Appendix: Critical appraisal checklist for health promotion programs

Introduction

This appendix has been adapted and updated from Bauman A & Rissel C (2003). Guidelines for journal reviewing. Health Promotion Journal of Australia 4(2):79–82.

Part A: Appraisal of individual interventions

1. Problem definition

• Is the health problem a public health priority? Is it a government or policy priority?

• What is the magnitude of the problem, and which population subgroups are at higher risk?

2. Formative evaluation and program development

• Is the problem amenable to change, through a feasible intervention? Does a literature search identify effective interventions? If there are several published studies, are there summaries of the evidence (non-systematic or systematic reviews); or formal pooled estimates from multiple studies (using statistical techniques of meta-analysis)? From this review, what effect size is generally observed for this intervention?

• Is there evidence of piloting or testing of the intervention or its component parts?

• Was the intervention tested with people similar to the proposed target group?

• Was there an underlying theoretical framework or conceptual model for the intervention?

• Were the intervention strategies and settings identified as ‘best practice’ (evidence-based) approaches?

• Are there funds for the intervention to be delivered as planned, and is there sufficient time for expected effects or changes to be observed?

• Was there a logic model to describe the ways that the intervention/program might work? (This will inform the priorities for which components warrant evaluation.)

3. Process evaluation

• Is there evidence of process evaluation to monitor the implementation, reach and participant acceptance of the program or its components?

• Is there evidence of adaptation of the intervention, fitting the intervention into different settings (and whether the effects were similar in different contexts)?

• How many people received (attended, participated in) the intervention? Were they typical of the target group (or were they different to those who did not participate)? Of all those who could participate or be included, how many actually did so?

4. Research methods to appraise the stages of impact and outcome evaluation

Study (research) design

• What was the study (research) design in this evaluation? Was it the most feasible study design that might have been used in this setting within available resources? Will the design provide sufficient rigour to demonstrate good-quality evidence of effectiveness?

Study sample

• What is the target population? Are the people who actually participated in the intervention typical of the target population? Were there any selection effects (e.g. non-representative/volunteer participants) that might influence the conclusions on program effectiveness?

• How were people recruited into, or how did they volunteer for, the study?

Measurement

• Are there specified and measurable intervention objectives?

• Are all relevant processes and outcomes assessed? How important are any omitted outcomes?

• Were the measuring tools, ...