Evidence-Based Practices Guide: Background

This page provides the text of the background to our “evidence-based practices tool”, available here.        

Introduction

The past decade has witnessed a tremendous expansion of research and resources on “what works” to improve adolescent health.  First, there are greater numbers of programs that focus on changing the contexts in which adolescents live, such as families, schools and communities as well as changing the individual behavior of adolescents, a more traditional focus.  In addition, the past decade has witnessed an increase in the creation of “implementing tools” – tools that help program managers and communities with critical program implementation tasks. Despite the greater selection of programs and tools, this valuable implementation information is difficult to locate in one place. This brief has two purposes.  First, by providing an annotated list of resources with links to evidence-based practices, it serves as a guide to communities and practitioners for locating effective behavioral/social interventions in adolescent health. Second, this brief also identifies “implementing tools” that are available from the listed resources for evidence-based practices. These tools serve as a useful starting point to help communities implement evidence-based programs, but are not intended to be an exhaustive list of all such tools.

What are Evidence-Based Programs?

As evaluation research has grown more sophisticated, the criteria for labeling a program as “effective” or “impactful” have become more rigorous.  As shown in the listings below, numerous researchers and organizations have developed criteria for what qualifies as an evidence-based program.  While a comprehensive review of these definitions is beyond the scope of this brief, methodological criteria have generally become stricter in areas such as study design (e.g., more randomized designs), sample size, effect size, statistical significance and replication of findings.

Evidence-based programs represent the “gold standard” along a continuum of what research supports as effective.  Many organizations also offer resources on “best practices” or “promising practices.”  These terms are generally applied to programs and strategies that have been evaluated with promising findings that do not meet more rigorous standards (e.g., programs that have some quantitative data from non-experimental studies showing positive outcomes in behavior or from a single experimental study that has not been replicated).  Sometimes these terms refer to consensus recommendations based on wisdom from professionals in the field, who consider available data and information when they make recommendations or develop guidelines for areas of a field without an adequate evidence base.  Many of the resources reviewed in this brief also contain best practices or promising practices.  Other useful resources, not reviewed in this document, offer best practices only and/or helpful implementing tools.

The resources on evidence-based programs use presented use different criteria for defining program effectiveness.  We do not endorse any particular definition of program effectiveness in this resource. Where possible, descriptions of resources listing evidence-based programs direct the user to each organization’s definition and/or criteria for these terms.

Using Evidence-Based Programs

Not all evidence-based programs are appropriate for all communities. Therefore, it is important to balance research about what works with knowledge about what is feasible in specific communities. Most of the resources of evidence-based programs described in this brief also provide or refer users to “implementing tools.”  These include comprehensive guidebooks and manuals (most of which are free). These implementing resources are noted in the resource descriptions, which usually contain links to downloadable documents. Although not a comprehensive review of all implementing resources, this brief provides a useful “starter set” for communities an`d program managers to consider. These tools address many issues that practitioners working with communities face as they design and implement evidence-based programs. These issues include:

• Identification of risk and protective factors that should be addressed (as determined by a needs-and-assets assessment);
• Selection of strategies that operate at multiple levels of intervention and have been shown to be effective with a community’s socio-demographic profile;
• Adaptation of an evidence-based program, while maintaining sufficient fidelity to the original program;
• Identification of programs that are feasible given existing resources – financial as well as local capacity and skills (e.g., communities need to consider training and capacity building that may be necessary to replicate an evidence-based program).

This brief recognizes that the identification of effective programs alone is not sufficient to ensure that programs are effectively replicated and appropriate for a given community or setting.  Reviewing evidence-based programs prior to designing an initiative gives immediate direction to program planning and can help prevent communities from wasting precious time and resources on ineffective strategies.  For this reason, funders increasingly place priority on using evidence-based programs.  Replicating and evaluating evidence-based programs contributes to further testing of effective strategies. For example, knowledge is gained from implementation and evaluation of a program with a different population, under different conditions, or with culturally appropriate modifications.

Limitations of Existing Evidence-Based Programs

Despite the greater range of evidence-based programs available compared to a decade ago, limitations remain.  For example, a large and growing body of research indicates that contextual influences, such as policies, families, schools and local communities, have a tremendous influence on adolescent health outcomes.  While the body of research on “what works” at community and policy levels has grown, gaps remain.  Due to the greater challenges inherent in measuring the impact of environmental change and policy-based interventions, there is a greater number of evidence-based programs focused on the individual level of change, particularly on behavior change by adolescents.  Despite the more limited research on interventions beyond the individual level, communities may decide to pursue multi-level approaches.

A community may also find that existing evidence-based programs have been demonstrated to be effective with a limited profile of socio-demographic groups and may not reflect the adolescent population it serves. For example, there has been an increase in the number of programs that have demonstrated effectiveness with Latino and African American youth, but less research has involved Asian/Pacific Islanders, a population with several distinct sub-populations. There are also few evidence-based programs for adolescents in rural settings.

In cases where no evidence-based programs meet the needs of a community, program managers still have options that are supported by research. By reviewing best practices and promising practices, communities can consider what adaptations are necessary to make an evidence-based program more relevant to its intended audience. For example, a community coalition aiming to reduce injury among Latino youth might decide to tailor an existing program for that population. It may help to research the types of interventions that have been effective in reaching Latino communities in general, and Latino youth in particular. It is also advisable for a community coalition to work with parents, youth and teachers to consider what, if any, cultural adaptations are needed. When tailoring interventions that have been shown to be effective, it is important to maintain a critical core from the original program by staying as true as possible to the evaluated model with respect to such key factors as learning objectives, the number of classroom hours, and types of activities – this is called maintaining fidelity.

In addition to limitations of scope and socio-demographic groups, there is a greater number of evidence-based programs in certain adolescent health areas than in other areas. More evidence-based programs exist to prevent teen pregnancy or substance use, for example, than to increase exercise or reduce injury. For emerging issues, such as adolescent sleep patterns or hearing loss, there is little evidence about “what works.” In such cases, communities need to examine best practices and evidence-based research in other health areas to review whether parallel findings in other content areas have implications for their own efforts. No matter what health issue is being addressed, adolescents need: specific knowledge regarding the issue; a specific set of skills that enables them to adapt and apply that knowledge to their own behavior; motivation to use those skills; a family, school and community environment that supports use of the requisite knowledge and skills; and a policy environment that provides sufficient resources and political commitment for improving adolescent health and changing social norms.

Organization of this Brief

This online version of the brief presents evidence-based program resources in two formats. First, a matrix provides an overview of the resources and includes links to each resource and indicates whether it includes implementing tools.  The health topics listed are adapted from the Focus Areas of the Healthy People 2020 Core Indicators for Adolescent and Young Adult Health.

The second section presents an annotated listing of each evidence-based program resource. Each listing begins with a description of the topics addressed, how the resource is organized, criteria for inclusion as an evidence-based program and what search functions are available. The listing also describes implementing tools (e.g., guidelines and manuals for needs assessment and program selection, working with communities) and other resources (e.g., fact sheets, “stories from the field”). Given the variation across these resources, not all descriptions will include each of these features. Links to the webpages are provided to the extent possible.

Implementing evidence-based programs is a complex undertaking. In addition to familiarity with evaluation research, it requires the capacity to collaborate with community partners and other stakeholders such as funders and supporting public agencies. We hope this tool helps guide practitioners to the tools they need to successfully use evidence-based programs to improve adolescent health.

To view the Evidence-Based Programs Resource Matrix and Annotated Listings, please click here. 

To download the PDF version of the brief, please click here. 

August 2014