Chapter 7: Evaluation methods for program replication, scale-up (dissemination) and institutionalisation

Introduction

Public health change requires reaching large proportions of the population with an intervention. This chapter is concerned with evaluation of efforts to test established interventions in new groups (replication studies) and to scale them up to even larger populations (scale-up or dissemination). This generally requires increasing the emphasis on process evaluation to understand how implementation of interventions work at a larger scale.

This chapter builds on Chapters 5 and 6, which described the evaluation of individual and comprehensive public health interventions, and is concerned with the stages of program replication, scale-up and institutionalisation, beyond the initial testing of whether a program works. These stages of replication, scale-up and institutionalisation are based on the need to deliver interventions and programs to larger and more representative groups. These are illustrated as stages 4, 5 and 6 in Figure 7.1 (which adds to Figure 2.1 in Chapter 2).

Health promotion interventions found to be effective, affordable and consistently capable of implementation are most likely to attract the attention of funders as suitable for policy support, scale-up and even institutionalisation. It is unwise to base a policy decision to scale-up a program and commit significant funding on evidence that is derived from one evaluation study, especially if results are equivocal or the effects are small. This is true even if the issue is a key priority, but evidence of successful interventions is scarce.

7.1 Stages in assessing the significance of programs

This chapter describes the three stages of replication, scale-up (dissemination) and institutionalisation. Across these stages, increasing numbers in the target population are exposed to and engage with the intervention. The three stages may seem obvious, but after years of working on solutions to public health problems, practitioners and researchers may lose sight of the population-wide goal represented by these stages. Put simply, policymakers and practitioners need more practice-relevant ‘evidence’ on interventions feasible for implementation at scale within available (constrained) resources and service configurations. Providing this evidence requires testing scaled-up program delivery using different evaluation methods to those typically used in researcher-led efficacy studies described in Chapter 5.

The model in Chapter 2 is re-drawn here as Figure 7.1 to identify the practitioner challenges in evaluating scale-up (demonstration, scale-up and intervention monitoring) and the researcher’s role in implementation science. Note that the third row of Figure 7.1 with light shading focuses on the role of implementation science, which is the purview of researchers, and is explained later in this chapter (see Box 7.1). The last row focuses on translational policy ...