Beyond Safe Havens

Amy J.A. Arbreton
Jessica Sheldon
Carla Herrera
Beyond Safe Havens:
Public/Private Ventures
A Synthesis of 20 Years of Research on the Boys & Girls Clubs
Executive Summary

Amy J.A. Arbreton
Jessica Sheldon
Carla Herrera
Beyond Safe Havens:
Public/Private Ventures
A Synthesis of 20 Years of Research on the Boys & Girls Clubs
Public/Private Ventures is a national nonprofit organization that
seeks to improve the effectiveness of social policies and programs.
P/PV designs, tests and studies initiatives that increase
supports, skills and opportunities of residents of low-income
communities; works with policymakers to see that the lessons
and evidence produced are reflected in policy; and provides
training, technical assistance and learning opportunities to
practitioners based on documented effective practices.
Board of Directors
Siobhan Nicolau, Chair
President
Hispanic Policy Development Project
Gary Walker
President
Public/Private Ventures
Amalia Betanzos
President
Wildcat Service Corporation
Yvonne Chan
Principal
Vaughn Learning Center
Mitchell S. Fromstein
Chairman Emeritus
Manpower Inc.
The Honorable Ren? Cardwell Hughes
Judge, Court of Common Pleas
The First Judicial District,
Philadelphia, PA
Christine L. James-Brown
President and CEO
United Way International
John A. Mayer, Jr.
Retired, Chief Financial Officer
J.P. Morgan & Co.
Matthew McGuire
Vice President
Ariel Capital Management, Inc.
Maurice Lim Miller
Director
Family Independence Initiative
Anne Hodges Morgan
Consultant to Foundations
Marion Pines
Senior Fellow
Institute for Policy Studies
Johns Hopkins University
Clayton S. Rose
Retired, Head of Investment Banking
J.P. Morgan & Co.
Cay Stratton
Director
National Employment Panel,
London, U.K.
William Julius Wilson
Lewis P. and Linda L. Geyser University
Professor
Harvard University
Research Advisory
Committee
Jacquelynne S. Eccles, Chair
University of Michigan
Ronald Ferguson
Kennedy School of Government
Robinson Hollister
Swarthmore College
Alan Krueger
Princeton University
Reed Larson
University of Illinois
Milbrey W. McLaughlin
Stanford University
Katherine S. Newman
Kennedy School of Government
Laurence Steinberg
Temple University
Thomas Weisner
UCLA
Beyond Safe Havens: A Synthesis of 20 Years of Research on the Boys & Girls Clubs
Acknowledgments
We would like to acknowledge the many people
who contributed their support, time and expertise
to make this report possible.
Boys & Girls Clubs of America (BGCA) provided
the funding for this report, and numerous BGCA
staff reviewed and commented on early drafts. In
particular, we appreciate the helpful suggestions of
Ed Mishrell and Karen McDonald, who coordinated
the feedback from other BGCA staff and communicated
it to us. In addition, BGCA provided the
photographs that appear on the cover of the report
(copyright 2001).
We would also like to thank the P/PV staff whose
contributions to the report were significant. Karen
Walker provided important suggestions for framing
the report and insightful comments that honed
its content and structure. Edward Moran wrote the
early draft of the executive summary and provided
valuable editing and copyediting services for both
it and the full report. Gary Walker reviewed drafts
of the report, Malish & Pagonis designed it, and
Chelsea Farley coordinated the editing process, as
well as the report?s publication and dissemination.
Beyond Safe Havens: A Synthesis of 20 Years of Research on the Boys & Girls Clubs
Beyond Safe Havens: A Synthesis of 20 Years of Research on the Boys & Girls Clubs
Executive Summary
Over the past two decades, Boys &
Girls Clubs of America (BGCA) has used a variety
of programmatic strategies that reflect shifting ideas
about what young people need to be successful and
which young people should be targeted for services.
In the 1980s and early 1990s, the focus of these programs
was on delinquency prevention to discourage
such behaviors as substance use or gang involvement.
From the late 1990s to the present, BGCA has prioritized
the integration of career preparation activities,
academic support and technology into its Clubs.
Importantly, throughout its history and despite shifts
in the policy and funding world, the BGCA mission
has emphasized providing all its programs within a
safe, fun and supportive environment.
To better understand the potential benefits of these
programs, more than 20 evaluations have been
commissioned over the past 20 years. Although
yielding promising findings, the vast majority of
these studies have been limited to documenting
only the specific outcomes the programs were
designed to influence (e.g., increased academic
achievement, reduced levels of drug abuse, etc.),
rather than considering that these programs are
part of the larger fabric of the Clubs? opportunities
for leadership, decision-making and positive peer
and adult-youth relationships?experiences likely
to affect broader outcomes for youth. Even the few
evaluations that have looked at club experiences
more broadly?while finding support for some
broader outcomes?have been limited in the outcomes
they explored, and none have looked longitudinally
at the Clubs? effects on youth.
At the request of BGCA?s national office,
Public/Private Ventures (P/PV) has undertaken a
multiyear study to understand the benefits that the
Clubs may have for the lives and outcomes of early
adolescents. As a prelude to this larger study and
to help hone its methodological focus, P/PV has
reviewed the range of studies that have been conducted
on Boys & Girls Clubs over the past 20 years,
all of which are covered in more detail in the full
report.1 The report covers three key areas:| It reviews the research conducted to date on discrete
BGCA programs to identify the potential
benefits of those programs for youth and discusses
three additional studies that examined the
broader club experience;| It outlines the strategies that contribute to the
Clubs? successes, as well as any challenges that
may have impeded more positive results; and| It identifies aspects of these evaluations that,
together, suggest ways in which the overall ?Club
experience? may influence young people?s health
and well-being?dimensions that have not been
fully explored in these studies.
The report concludes with a brief description of a
planned longitudinal evaluation of Club members
as they transition to high school?an evaluation
meant to provide documentation of the effectiveness
of teens? broad Club experiences on a wide
range of outcomes.
Twenty Years of Research on BGCA:
Findings
The standard operation of Clubs has made it difficult
for evaluators to carry out the most rigorous
kind of impact studies, in which youth are randomly
assigned to treatment or control groups. Traditionally,
Boys & Girls Clubs welcome everyone through
their doors and give youth great leeway in determining
the extent to which they will participate.
The willingness to accept all comers, which is part
of the BGCA mission, makes random assignment
impossible. Doing so would change the very nature
of the Club experience. Given this challenge, the
evaluations examined for this report used different
strategies to learn about program effectiveness,
none of which are without flaws. With this caveat in
mind, however, it is possible to ascribe important
conclusions to the existing literature, based on the
numerous studies of discrete programs, the few
studies that examine general Club participatory
experiences, and the implementation lessons that
emerge when all the studies are examined in sum.
2 Beyond Safe Havens: A Synthesis of 20 Years of Research on the Boys & Girls Clubs
Moreover, the studies? findings lend insight into
what additional, broader outcomes?as yet unexplored?
are likely affected by Club participation
more generally and deserve attention.
Detailed more fully in the report, the evaluations
found several outcomes linked to participation in
discrete BGCA programs, including:| Reduction in delinquent behaviors. Both prevention
classes geared toward all Club members
(e.g., SMART Moves, BGCA?s signature alcohol,
drug and pregnancy prevention program)
and targeted outreach models of delinquency
prevention that involve recruiting youth with
specific risk characteristics to engage in broader
Club activities (e.g., Gang Prevention through
Targeted Outreach) show evidence of reducing
youth?s participation in delinquent behaviors.| Increased academic achievement. Studies of
educational programs designed to integrate highyield
learning activities throughout the Club and
to offer homework help (such as Project Learn
and its pilot, the Educational Enhancement
project) document increased levels of homework
completion, involvement in discussions about
books and other discussions with adults, and
achievement in several academic areas.| Increased access to and safe utilization of
technology. Operation Connect was designed to
provide technology centers in Clubs to increase
youth access to computers and the Internet.
Findings indicate that children increased their
level of computer use and technology skills and
began to use the computers for homework. A
related program, NetSmartz, helped youngsters
become more aware of safety issues on
the Internet.| Increased career goals and improved attitudes
toward school. Two career-oriented initiatives,
Broader Horizons and Career Prep, reached and
retained a large number of youth in their early
teen years. Youth reported that the program
helped them set career goals and improved their
attitudes toward school.
Although none of the three more broad-based
studies reviewed for the report followed youth over
time, the evaluation findings suggest a range of
positive experiences:| Positive relationships and safe places to belong,
participate and contribute. Club members who
responded to a one-time survey and respondents
in two alumni studies overwhelmingly
reported having positive relationships with Club
staff. They also reported other benefits from
Club participation, including getting along
with others, having a safe place to interact with
others, feeling a sense of belonging and being
afforded opportunities for decision-making,
leadership and community service.
The studies suggest correlates of these positive
experiences:| Participation, tenure at the Clubs and connections
with staff. The extent to which youth and alumni
reported positive experiences and benefits from
the Clubs was associated with both tenure at the
Club and how well they felt they knew or connected
with staff. The variety of informal and formal
programs and opportunities the Clubs offered,
as well as the efforts to provide a rewarding and
youth-centered environment, were similarly found
to be related to teens? positive experiences, participation
and retention in these studies.
In reviewing the numerous evaluations of discrete
BGCA programs, the full P/PV report identifies and
describes implementation practices that appear to
promote the success of those programs:| Investment in developing caring relationships
between youth and staff. First and foremost,
across all the studies reviewed, caring relationships
between youth and staff were identified by
youth, parents, Club staff and partner-agency staff
as crucial to the success of the program?they
were why the youth came and why they stayed
in the program. This is in keeping with numerous
studies that identify supportive relationships
in youth?s lives as a key factor in promoting
resilience and healthy development. Clubs most
successful at facilitating these supportive relationships
gave staff sufficient time for working
Executive Summary
one-on-one with youth and ensured that enough
staff were devoted to the program to provide sufficient
support for youth.| Targeted recruitment strategies. A primary goal
of many of the evaluated programs was to recruit
youth?particularly high-risk youth?to participate.
Three strategies for recruitment and retention
were particularly effective: street outreach, referral
from schools and families, and partnerships
with community agencies.| Interesting and exciting programming. Creating
programs that youth found interesting and enjoyable
was key to retention. Successful strategies
employed by Club staff included flexibility in
tailoring the program to the needs and schedules
of youth, creating ?fun? activities and providing
rewards and incentives for participation.| Partnerships with other organizations, such
as schools, the police and justice agencies.
Although Clubs found partnerships with community
institutions challenging to build and maintain,
when successful, they provided significant
benefits for referral and information sharing.
Successful partnerships typically emerged when
Clubs clarified and emphasized the value of their
program(s) and devoted staff time to developing
and maintaining relationships.
Finally, P/PV?s review identifies significant implementation
challenges that traversed the study findings,
primary among them:| Staff turnover. This was a particularly troublesome
area for many of the programs, as high
levels of staff turnover often diminished a Club?s
capacity to provide quality programming and
forge lasting relationships. Because of the crucial
role of staff-youth relationships, strategies for
staff retention were also critical. Clubs increased
program effectiveness by recruiting staff with specific
skills (e.g., with educational backgrounds or
technological savvy), promoting staff from within
the club and taking steps to ensure staff buy-in
for any new programming.
Taking A Broader View of the Role of
Clubs
Though the majority of the studies reviewed for this
report looked at specific outcomes, such as changes
in drug use, academic achievement and career
goals, Club members appeared to derive additional
benefits from the Club experience beyond what was
measured by the studies? discrete outcomes. Several
of these benefits were touched on in the three
broad-based studies. Young people were having fun,
interacting positively and constructively with adults,
engaging in activities that were building their competencies
and confidences and interacting with
peers in positive ways. By looking more closely at
these findings and examining the implementation
lessons that traversed them, the P/PV report reveals
a host of other areas where Clubs seek to impact
members? lives, including a positive view of the
future, commitment to school and learning, integrity
and self-efficacy.
The broader potential of Clubs to provide these
types of experiences and positive outcomes for
members?those that cut across several domains?
has not been explored fully in any of the studies
conducted to date but is critical to test in order to
gain a solid understanding of the role that Clubs
play in the lives of the youth they serve. This must
include a longitudinal investigation to examine how
these Club experiences support youth at critical
junctures in their lives.
As the full report describes in more detail, a longitudinal
study that broadens outcomes to match the
Club?s mission and strategies and that builds on the
findings from the past 20 years will appropriately
test the role of the Clubs ?as a whole experience?
in the lives of the adolescents they serve. A study of
this nature, designed to learn about the relationship
of these Club experiences to the teens? emotional,
behavioral and academic health, will also have implications
for other organizations that strive to support
positive developmental trajectories for youth.
Endnotes
1 A copy of the full report is available online at www.bgca.org or
www.ppv.org.
Beyond Safe Havens: A Synthesis of 20 Years of Research on the Boys & Girls Clubs

September 2005
Public/Private Ventures
2000 Market Street, Suite 600
Philadelphia, PA 19103
Tel: (215) 557-4400
Fax: (215) 557-4469
New York Office
The Chanin Building
122 East 42nd Street, 42nd Floor
New York, NY 10168
Tel: (212) 822-2400
Fax: (212) 949-0439
California Office
Lake Merritt Plaza, Suite 1550
1999 Harrison Street
Oakland, CA 94612
Tel: (510) 273-4600
Fax: (510) 273-4619
http://www.ppv.org