Equity-focused Strategies in a Federally Funded Evaluation of a Sexual Health Program

Child Trends collaborated with Equimundo and the Latin American Youth Center (LAYC) to lay the groundwork for a successful and equitable evaluation of MWB. Each partner played a unique and critical role in advancing equity in evaluation planning and preparation. In this brief, we highlight key equity-focused strategies used within our first year of funding in three central project preparation activities led by a different program partner: program design and adaptation, program implementation, and evaluation. We hope the approaches described throughout this brief can generate ideas for other federally funded programs as they begin their evaluations.

---

Equity-Focused Strategies By Partner Role in the Many Ways of Being Program Evaluation

---

Key equity strategies

Below, we summarize the key equity-focused approaches described in more detail throughout this brief. For each activity, we highlight the program partner that led that effort.

Program design and adaptation: Equimundo

Equimundo redesigned the MWB program to prioritize youth’s desires and needs. Specifically, they partnered with Black and Latinx youth to adapt the program to better reflect the lived experiences of youth,  honor intersectional identities, acknowledge historical harm by the medical community, and incorporate content on the influence of media on youth today.

Program implementation planning: Latin American Youth Center

Latin American Youth Center prioritized reciprocity and the needs of the youth and schools. Specifically, they:

• Identified and met the needs of both schools and students in partnering on program implementation, and shared results data back with schools.
• Advocated to the project team—on behalf of the project’s target population of Black and Latinx students in the Washington, DC metropolitan area—based on the experiences and knowledge of community facilitators.
• Implemented hiring practices to bring together a team of facilitators with multiple identities and diverse lived experiences who understand the project’s target population of youth, as well as their role in prioritizing equity and justice through facilitation.

Program evaluation planning: Child Trends

Child Trends explored internal biases and took a strengths-based approach to evaluation. Specifically, they:

• Conducted culturally responsive and equitable assessments of the project team and held regular and ongoing team reflection meetings.
• Aligned evaluation surveys and qualitative protocols with a strengths-based and inclusive framework.
• Provided a high-quality and beneficial control program to meet community needs.

---

Program Design and Adaptation: Equimundo

Equimundo’s approach

In designing the MWB program, Equimundo drew from two existing programs, Manhood 2.0 and Sisterhood 2.0. These programs aimed to help students identify gender norms that operate in their community, observe the consequences of inequitable gender norms, and explore the benefits of reimagined (more equitable) gender norms. However, offering single-gendered spaces was not inclusive of all identities and did not meet the needs of youth participants. To ensure that the program would be relevant to youth’s needs, Equimundo partnered with Healthy Teen Network to conduct formative research.

Using a human-centered design approach, Healthy Teen Network conducted a hybrid of ethnographic interviews and digital focus groups to learn from 1) non-cisgender and cisgender Black and Latinx youth ages 15 to 19, and 2) former facilitators from similar sexual and reproductive health programs. The goal of getting feedback was to better understand the priorities and needs of the populations that will be served through the program and to redesign the program with those needs in mind.

Specific program design and adaptation strategies

In response to formative research findings, Equimundo strengthened the core content and methodologies by promoting inclusivity, addressing intersectionality, discussing media and technological contributions to messaging, and adapting to youth’s learning preferences.

Promoting inclusivity of all genders and sexual orientations

Equimundo ensured that the MWB program was representative of non-heterosexual relationships and sexual orientations and incorporated discussion of gender identities beyond the male/female binary. For example, in many roleplays and scenario examples throughout program sessions, characters have a variety of pronouns and sexual orientations. This not only allows students to become familiar with diverse gender identities and sexual orientations, but also to think about the ways in which these multiple identities may influence the situations presented through roleplays or scenario activities. Other examples from the program include:

• Encouraging students to share their pronouns with the group and use others’ preferred pronouns.
• Including the Gender Unicorn, an external resource created by Trans Student Educational Resources that graphically illustrates the various dimensions of sexuality and identity.
• Using the term “people with penises” instead of boys/men during an activity related to STIs, as this may be an important distinction for trans or nonbinary people.

Emphasizing and addressing intersectionality

The MWB program considers various identities—such as age, wealth, disability, geography, immigration—that may impact young peoples’ lived experiences. The program incorporates elements of intersectionality into program activities and content by:

• Using group discussion questions that probe for differences in experiences based on age, race, ethnicity, wealth, and a variety of other identities—an approach particularly evident in the sessions on contraceptives and access to health care.
• Providing opportunities for reflection on medical racism and how it might impact Black and Brown communities’ history and access to health services.
• Incorporating roleplays about power in relationships and using a variety of personal characteristics to explore the intersectional dynamics of power and privilege. These dynamics are further explored in an activity that demonstrates how pornography often perpetuates damaging stereotypes of minority identities (racial, gender, sexual orientation, etc.).

Incorporating content on the role of media and technology in influencing messages about gender, sex, and sexuality

Focus group participants emphasized that any program that addresses sexual and reproductive health must also address modern media, including pornography and sexting. Therefore, the MWB developers included an activity that encourages youth to reflect on the subtle and overt messages about relationships and sexuality that they absorb in social media and advertising. Youth are also provided an opportunity to critically think about and openly discuss the purpose, benefits, and risks associated with pornography and sexting.

Prioritizing youth-requested learning modalities

Focus group participants indicated that they did not want a program that simply provided information they could find online; instead, they wanted to learn skills and have opportunities to apply those skills. Therefore, the MWB developers adapted existing activities to include more opportunities to apply the knowledge gained during program sessions through roleplaying scenarios and case studies. The formative research for this project reinforced that youth connect with one another predominantly in digital spaces (texting, social media, etc.). Therefore, the MWB team designed a complementary digital component, including a set of 34 ready-to-go Instagram posts and stories that reinforce the program’s messaging and create a digital space for participants to further engage on program topics.

---

Program Implementation Planning: Latin American Youth Center (LAYC)

LAYC’s approach

In preparing to implement the first year of the evaluation, LAYC focused on hiring and building an implementation team of facilitators, as well as recruiting and developing relationships with partners. LAYC relied on existing partnerships and connections to create awareness about the program and connect with potential partners.

Specific program implementation strategies

Throughout implementation, LAYC embedded equity into the project by prioritizing reciprocal relationships with implementation sites, knowing and advocating on behalf of youth participants’ needs, and hiring staff with community- and population-relevant experience.

Prioritizing reciprocity in relationships with implementation sites

LAYC invested in strong and lasting relationships with several schools in the Washington, DC metropolitan area by identifying and strategizing ways to meet students’ and schools’ needs. Prior to meeting with schools, LAYC staff reviewed websites and, when possible, scheduled informational interviews with LAYC staff in different departments who may have interacted with the schools previously to learn about the schools and their students. LAYC sought to partner with schools that were not currently offering sexual health programming at the time of partnership development; the organization intends to continue working with schools to implement MWB even after the project ends to help meet these schools’ need for sexual reproductive health programming. From the beginning, LAYC consistently asked school staff about their needs and requirements and prioritized reciprocity by offering to provide a results document to school partners to share outputs and outcomes during implementation.

Understanding and advocating on behalf of the young people being served

LAYC has been embedded in the Washington, DC community for 55 years and has extensive experience engaging youth with a variety of racial and cultural backgrounds and diverse identities. As such, LAYC staff provided input throughout the program development process, particularly to ensure that the language used was inclusive of nonbinary and trans youth. Further, LAYC raised the idea of—and advocated for—conducting programming in Spanish because the Washington, DC area has a large population of Latinx youth. As a result, Equimundo translated the program and all evaluation materials into Spanish to better serve the youth in the community.

Hiring staff embedded in the community being served and who have a strong understanding of social justice and equity

As the implementation partner and organization working directly with youth for the MWB project, LAYC hired staff and program facilitators who understood the needs of the populations being served. LAYC focused on local candidates who understood the challenges and issues faced by Black and Latinx youth in the Washington, DC metropolitan area. For example, LAYC hired two curriculum facilitators who had previously participated in other LAYC programming. Further, when interviewing staff, LAYC asked questions to ensure that facilitators had considered and understood their role in promoting social justice and equity through facilitation:

• How might/do your multiple identities influence your work as a facilitator?
• What do you think your role is in social justice efforts?
• What does equity mean to you?

---

Program Evaluation Planning: Child Trends

Child Trends’ approach

Child Trends staff prepared to evaluate MWB by assessing their own evaluation team’s internal biases and ensuring that the evaluation design was strengths-based, culturally appropriate, and responsive to the needs of Black and Latinx teens in Washington, DC.

Specific program evaluation strategies

To prepare to evaluate MWB, Child Trends evaluators assessed the research team for cultural responsiveness and biases, matched data collection protocols to participants’ needs, and provided a high-quality control program.

Holding culturally responsive and equitable self-assessments and team conversations

To encourage an open dialogue around equity, the Child Trends team completed culturally responsive evaluation self-assessments and discussed their responses, with the goal of furthering equity practices in the evaluation. This activity allowed the team to critically think about practicing cultural responsiveness and to examine potentially interfering biases at an individual and group level. These conversations also inspired discussions with LAYC and Equimundo around what success in this study would mean to the team and to the target populations. As a result, the research questions have shifted away from a deficits-based, prevention focus and toward a sexual health promotion and empowerment focus. This ensures that the outcomes of interest align with agreed-upon benchmarks of success and are appropriate, responsive, and equitable to the populations we aim to serve (more below).

Aligning asset-based data collection protocols with the needs and lived experiences of target populations

An important component of equity in evaluation is developing questions to assess program impacts that are meaningful to the stakeholders. The MWB program takes an asset-based approach and has shifted the framing of program content away from deterring youth engagement in “unhealthy” or “negative” sexual behaviors and toward encouraging youth to engage in positive, healthy sexual behaviors. The evaluation team built the participant survey and focus group protocols to align with this ideology. For example:

• The primary research question assesses MWB’s impact on reducing unprotected sex instead of reducing sexual activity generally; this helps avoid shaming youth who choose to be sexually active. Further, the team developed secondary research questions and study measures to understand how the program promotes sexual agency,[i] self-efficacy to engage in healthy relationship behaviors,[ii] and sexual and reproductive empowerment.[iii]
• Survey items include answer choices that reflect the lived experiences of youth in the program and exclude items that could be considered harmful or language that may appear shaming. For instance, Child Trends received a waiver from the Institutional Review Board to remove a federally required binary gender question—“What is your sex? Male; Female”—and replaced it with an inclusive gender question: “Which of the following best describes your gender identity (which may be different than your sex assigned at birth)?​ Female (Girl/Woman)​; Male (Boy/Man)​; Non-binary, Gender fluid, or Gender expansive​; A gender not listed here.” Additionally, Child Trends removed unnecessary questions about life context that could be harmful to youth—for example, questions about neighborhood and school violence. These data are unnecessary to accurately assess the impact of MWB and including these types of negative questions in the survey can be discouraging to youth.
• Child Trends revised focus group protocols with a strengths-based framing. For instance, the question “What are the biggest issues that teens face?” now asks “What are the most important things teens need to thrive?”
• Child Trends conducted cognitive interviews to test the cultural appropriateness and accessibility of the sexual and reproductive health measures with Black and Latinx teens in Washington, DC.

Providing a high-quality and beneficial control program as part of the evaluation

To ensure that the evaluation design is centered on participants and the needs of the priority community, the project team set aside resources to implement high-quality and beneficial comparison programs. One example includes the International Youth Foundation’s Passport to Success® (PTS) program. By offering a high-quality comparison program, youth who are not randomly selected to receive MWB still receive benefits from participating in the evaluation. PTS equips young people with a range of skills to help them stay in school and acquire the education, professional skills, employment readiness, and confidence they need to succeed in life and in the workplace.

 

---

Conclusion

Child Trends, in collaboration with its partners at Equimundo and LAYC, used the evaluation planning year to align MWB’s program, implementation, and evaluation design with key equity principles and to lay the groundwork for a successful and equitable evaluation. Each partner played an important role in this process, highlighting the advantage of a collaborative and multipronged effort to advance equity.

Following the project planning period, the team piloted the MWB program in Fall 2022 and began full implementation in 2023. Since then, the team has continued to prioritize equity in design, implementation, and evaluation by updating the implementation and evaluation approach based on evaluation findings and input from program participants and staff. For example, Equimundo updated the MWB program to better respond to the needs of youth participants and school staff by 1) translating it into Spanish to accommodate Spanish-speaking participants and 2) creating a version of the program with 16 one-hour sessions (instead of eight two-hour sessions) to allow for in-school implementation. To create a peer-peer approach for recruitment, LAYC asked interested program alumni to serve as youth ambassadors. In this role, youth ambassadors support program recruitment by discussing their experiences in the program with their peers. To guide future project improvements, the team created and continually updates a “lessons learned” document to report on what is working well and what can be improved about the MWB program, as well as implementation and evaluation practices.

Although the team incorporated equity principles into the MWB project during the project planning period and continues to make changes to further promote equity, we have identified several areas of opportunity for improvement in future work. For example, a key component of the program and evaluation design processes focused on ensuring that the program is inclusive of students of all genders and sexual orientations. However, through the pilot, the team noted that the programming and implementation could be more supportive of other youth populations, including youth with learning differences or challenges. We have addressed this in part by researching, discussing, and implementing best practices on supporting youth with intellectual and developmental disabilities—for example, by allowing youth to take additional breaks, providing an extra adult in the room to support the youth, and having direct discussions with the youth on how facilitators can best support them. We recognize, though, that we can do even more in future work—for example, by providing staff trainings on serving youth with learning differences or disabilities and partnering with school counselors to gain insight on how to better to support youth.

Another important component of equity-focused evaluations of sexual and reproductive health programs for youth is to intentionally incorporate youth voices into all processes. The MWB evaluation has incorporated youth participants into formative research to inform program design, recruited youth ambassadors to help with recruitment and retention, and conducted youth focus groups to obtain information about student responsiveness to the program and facilitators. However, future work would ideally incorporate youth voices in all stages, including selection of research questions and evaluation design. In later stages of the evaluation, we plan to engage youth and school partners in interpreting evaluation findings through brief presentations and data walks. During data walks, we will ask small groups of students or school partners to review study findings and then participate in a discussion to interpret, discuss, and reflect on these findings.

While we have presented strategies for federally funded programs to advance equity through evaluation, we recognize that funders are critical for any program to advance equity. The PREIS grant incorporated an up to 18-month planning period that allowed the team to adapt the MWB program, partner with implementation sites, and design and pilot the evaluation. We recommend that funders continue incorporating a similar planning period within evaluations to allow project teams time to design an equitable evaluation. It’s important to have funders who champion diversity, equity, and inclusion (DEI) in evaluation work and who are willing to fund a role focused on DEI to better support equitable implementation and evaluation. We also understand that including youth voices in all stages—including to provide feedback on research questions—would require additional funds and flexibility from funders. Lastly, while federal program evaluations are designed to expand the pool of evidence-based programs available to communities, researchers must go beyond incorporating equity into evaluation to incorporate equity and inclusion in program development. As a result, we recommend funding innovation development that incorporates youth and community voices to ensure the development of programs that are responsive to the needs of communities.

---

Acknowledgments

This brief was made possible by Grant Number 90AP2693 from the Department of Health and Human Services, Administration for Children and Families. Its contents are solely the responsibility of Child Trends, Equimundo, and LAYC and do not necessarily represent the official views of the Department of Health and Human Services, Administration for Children and Families.

As authors, we would like to thank our colleagues, without whom this work would not have been possible. First, we would like to thank our wonderful program facilitators Altagracia Kubinyi, Dereck Myers, Alma Figueroa, France Robinson, and Sokka Asif. We also would like to thank our former project officer, Kathleen Derrick, for thought partnership throughout the project planning period; our current project officer, Katherine Godesky, for her review of this brief; Lauren Kissela for her fact check; Brent Franklin for his edits; and Catherine Nichols for her design work. Finally, we are deeply grateful to the staff at our implementation sites and to our youth participants, who offered their time and perspectives—all of which were critical to making the MWB evaluation project stronger.

---

---

Endnotes

[i] Expressed sexual agency items used in the surveys were developed by the Child Trends team based on conversations with Equimundo and researcher Spring Cooper. Perceived sexual agency items were adapted from the Female Sexual Subjectivity Inventory and edited to better fit the age and gender diversity of the MWB evaluation sample.

[ii] Self-efficacy items were adapted from the Re:MIX program evaluation survey and the Access to Sexual and Reproductive Health Information measure in Upadhyay et al.’s Sexual and Reproductive Empowerment Scale for Adolescents and Young Adults.

[iii] Sexual empowerment measures were adapted from Upadhyay et al.’s Sexual and Reproductive Empowerment Scale for Adolescents and Young Adults.