Skip to main content
Download PDF
Download ePub

Hmong Promoting Vaccines eHealth website: a community-based participatory research pilot to evaluate dissemination and implementation strategies for primary care and educational contexts

Implementation Science Communications volume 6, Article number: 45 (2025) Cite this article

Abstract

Background

Human papillomavirus (HPV) vaccination rates among Hmong American adolescents are significantly lower than national averages, despite higher cervical cancer rates in this population. eHealth can improve vaccine uptake, especially in the COVID-19 era. Our community-based participatory research team developed and evaluated a culturally-tailored website (https://hmonghpv.com) featuring educational modules on HPV and HPV vaccines in Hmong and English for Hmong adolescents and their parents. This pilot study aimed to determine the most effective dissemination and implementation (D&I) strategies within schools and primary care clinics using community-engaged methods, including community-based participatory research (CBPR), an understudied area in D&I research. CBPR has been underutilized in D&I research, yet its participatory approach ensures that marginalized voices are included, offering valuable insights for implementing and sustaining culturally adapted interventions to enhance uptake and long-term impact.

Methods

Our CBPR team included Hmong researchers, university researchers, a public health advocate, and a family medicine physician-researcher. We conducted pre- and post-implementation interviews with six user organizations and seven disseminator community-based organizations to guide the development and assessment of D&I strategies for the website. Using template analysis, we analyzed the pre-implementation data. We then partnered with the user organizations to co-design individualized D&I plans that they implemented during an eight-week pilot period. We used Google Analytics and a Qualtrics survey to assess website use post-implementation.

Results

During implementation, user organizations promoted the website to 300 new users, who spent an average of nearly 12 min on the site. The most robust dissemination (n = 117) occurred in a Hmong charter school that integrated the website into their health education curriculum. Post-implementation interviews revealed that D&I plans that fit into clinic workflows and school curricula had the most robust implementation, and that clinic staff found discussing HPV vaccines most useful during adolescent preventive health visits. Challenges included time constraints, lower receptivity to vaccine conversations at non-preventive visits, and adolescents’ preference for alternatives to paper handouts. Disseminator organizations showed strong interest in scaling the website for greater reach in Hmong and non-Hmong populations.

Conclusion

This pilot study demonstrated the feasibility of disseminating and implementing a culturally-tailored educational website for HPV education in educational and primary health care settings. Health education curricula in culturally-specific schools proved to be the most effective dissemination modality. A dissemination toolkit with support materials is available to facilitate using the website within educational and primary care contexts. Additionally, using a CBPR approach provided valuable implementation insights to enhance the intervention’s contextual validity and sustainability.

Peer Review reports

Background

Cervical cancer poses a significant public health concern among Hmong American women, who experience cervical cancer diagnosis rates that are more than four times higher than those of non-Hispanic White Americans and three times higher than other Asian American women (e.g., Chinese, Vietnamese, and Korean) [1]. The uptake of the human papillomavirus (HPV) vaccine can significantly decrease the occurrence of such cancers [2]. However, HPV vaccination rates among Hmong American adolescents are low relative to the general adolescent population. A 2015 review of medical records for Hmong American youth between the ages of nine to 17 years at a community health center in Minnesota revealed completion rates of 32% for girls and 20% for boys, which were significantly lower than the national completion rates of 57% and 35% at that time, respectively [3]. This disparity highlights the urgent need for strategies to promote HPV vaccination among adolescents within the Hmong American community.

The low uptake of HPV vaccines among Hmong Americans can be attributed, in part, to limited awareness and knowledge regarding the effects of HPV and the benefits of the vaccine among adolescents and parents [3]. This lack of awareness stems in large part from structural barriers at the healthcare system and societal levels. Similar to other recent Asian immigrants, Hmong Americans encounter multiple obstacles when accessing healthcare and health information, including language and health literacy barriers, gaps in insurance coverage, difficulty navigating a complex health care system, and challenges related to time and transportation [4, 5]. It is crucial to address these barriers in order to improve HPV vaccination rates within the Hmong American community.

Utilizing a community-based participatory research (CBPR) approach, our community-academic partnership designed a bicultural, bilingual HPV educational website (Hmong Promoting Vaccines, www.hmonghpv.com) [3]; the website features five educational modules that include didactic content and videos and covers topics such as the natural history of HPV and the benefits of HPV vaccinations. We piloted the website with 30 Hmong American parent-adolescent dyads to address these barriers, which showed that completion of the website modules significantly improved both parent and adolescent HPV and HPV vaccine knowledge [6]. Further, we observed increases in adolescents’ awareness and concerns about HPV-associated cancers, parents’ understanding of family, community, and healthcare factors related to HPV vaccination, and adolescent and parent health-seeking and communication behaviors related to HPV vaccination. Both Hmong parents and adolescents reported increases in collaborative decision-making and self-efficacy regarding HPV vaccination, and they found the website to be highly user-friendly and beneficial.

Building on the encouraging results of the feasibility study, this dissemination and implementation (D&I) pilot study aimed to engage community-based organizations to co-create and examine delivery strategies to facilitate greater reach of the Hmong Promoting Vaccines website among Hmong American parents and adolescents. A significant gap remains in the literature between the development of evidence-based health interventions that show promise for improving health outcomes in minoritized communities and the uptake of interventions in these communities [7]. Conducting dissemination research early on in the development of an evidence-based intervention to develop and test delivery strategies either before or alongside an efficacy or effectiveness trial is essential to enhance an intervention’s relevance, reach, and sustainability within historically underserved communities [8] and to enhance its external validity within these groups [9]. This early examination allows for a clearer sense of the contexts in which the intervention is likely to be used and how the intervention and its delivery strategies will need to be adapted for these conditions [10]. In D&I research, it is essential to engage potential recipients of an intervention (i.e., end users) and incorporate their input on dissemination. This approach helps assess organizational capacity for sustainability and ensures that delivery strategies are contextually appropriate [11, 12].

Policy makers, funding agencies, institutions, and experts in implementation science have called for involvement of community representatives across the translational research spectrum as an essential component of advancing health equity in minoritized groups. [11, 13, 14] Community engagement across the translational continuum through iterative and collaborative approaches can help to identify and design implementation and dissemination strategies that are conducive to the needs and interests of communities throughout the intervention and implementation processes [8, 12]. Co-creation of both the intervention and dissemination strategies in this way that are proactively tailored for specific communities may increase the uptake, acceptability, reach, and effectiveness of health interventions [8, 15]. Community partnership in research may also further reduce the barrier of mistrust that often exists between academic researchers and minoritized communities, leading to increased engagement in research and enhanced dissemination and sustainability [13]. Community engagement in implementation research can take on many forms ranging from more limited stakeholder engagement of a community advisory board up to CBPR, which involves community-based researchers as equal partners in the research process, leveraging their expertise and lived experiences in the research approach from conception of the research question to dissemination and implementation [15].

The use of CBPR approaches in D&I research is growing and offers the opportunity to expand the reach of evidence-based interventions (EBIs) to minoritized groups [16] including through guiding EBI adaptation for specific groups [17,18,19]. However, a review of recent National Institute of Health implementation science grants found that community-engaged, and especially community-partnered, approaches are still underutilized in health equity D&I research [20]. Additionally, much of the D&I science to date has focused more on adaptation of previously developed EBIs as the strategy to address health inequities rather than engaging community members early on in the design, testing, and dissemination of interventions that are developed specifically for the communities experiencing the health inequities [8].

Implementation science must reimagine its knowledge generation practices to be more inclusive, participatory, and equity-driven. Traditional top-down approaches risk reinforcing inequities by failing to consider the lived experiences of historically marginalized communities, limiting the real-world uptake of evidence-based interventions. Engaging community-based organizations and end users early in dissemination research—not just after feasibility and efficacy trials—bridges the gap between intervention development and implementation [7]. This co-creation process ensures that minoritized voices are not passive recipients of knowledge but active contributors, aligning with CBPR and equity-focused implementation frameworks [8]. Moreover, knowledge generation must extend beyond external validity to embrace “contextual validity,” adapting interventions and delivery strategies based on community input to improve cultural and logistical feasibility [9, 10]. By embedding dissemination within community engagement, researchers can also assess organizational capacity for sustainability, ensuring that interventions align with local resources rather than imposing one-size-fits-all solutions [11, 12]. Ultimately, prioritizing minoritized perspectives through community-engaged methods dissemination research transforms how knowledge is produced, making it more relevant, actionable, and effective in real-world settings.

In this paper, we build on this emerging body of D&I studies of culturally tailored interventions developed in partnership with minoritized groups using community-engaged research approaches [21, 22] to share an example of a D&I pilot study of our eHealth website developed with Hmong American adolescents and parents using CBPR methods. Our pilot study aimed to determine the most effective dissemination and implementation strategies for the Hmong Promoting Vaccines website within schools and primary care clinics.

Methods

Study team and approach

Our CBPR team consisted of three Hmong researchers, two university researchers, a public health advocate, and a family medicine physician-researcher, most of whom were involved in the initial development of the Hmong Promoting Vaccines eHealth website. The CBPR team was involved in every stage of the research process, from grant writing and study design to recruitment, data analysis, and dissemination. Following the initial success of the website in our pilot evaluation study, the team sought to expand its reach within the larger Hmong community and to mainstream organizations serving Hmong members. Together, we identified an internal grant at the University of Minnesota to support further dissemination and translation of the Hmong Promoting Vaccines website. After securing the grant, the team co-created a list of potential partner organizations to inform the website's implementation and dissemination. Drawing on the lived experiences and cultural knowledge of our Hmong researchers, we compiled an extensive list of trusted community-based organizations for collaboration. During recruitment and analysis, we ensured that a Hmong researcher led or participated in the process, and we made sure Hmong researchers were the primary point of contact and the face of the CBPR team throughout our dissemination and implementation efforts. In parallel with our CBPR work, we also formed a community advisory board to deepen community input in our D&I pilot study. The five-member community advisory board (CAB), composed of Hmong-identifying professionals from health care, public health, and educational backgrounds and one high school student, met with the team before and following the pilot study. Educational Technology Innovations (ETI), a software developer vendor from the affiliated university, also assisted the team in making post-hoc changes to the website.

Theoretical framework and D&I approach

Given our two-pronged D&I approach, we used the University of Washington Health Promotion Research Center’s (HPRC) Dissemination Framework developed by Harris and colleagues [23] to inform our D&I activities. This pragmatic framework provides a guide for researchers and disseminating organizations to partner in disseminating chronic disease prevention interventions to user organizations. In this model, researchers work closely with disseminators (i.e., organizations working to promote the use and spread of the intervention) to optimize and assess the D&I plan. Then disseminators take the lead in carrying out the plan and user organizations (i.e., organizations implementing and using the intervention) carry out the best practices to disseminate the intervention through implementation. Drawing on this framework, we divided our D&I activities into the following three phases: pre-implementation, implementation, and post-implementation. We then used the HPRC dissemination framework to guide the selection of our D&I partners (user organizations and disseminator organizations) and the development of study tools to obtain these partners’ input during pre- and post- implementation study activities as described below. We also used the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) Framework [24], an implementation science framework, to plan and evaluate our dissemination and implementation activities at each site. This framework is particularly valuable for assessing health intervention implementations in real-world settings [25]. The RE-AIM framework guided the selection of implementation outcomes (e.g., reach, evaluation, adoption, implementation, and maintenance) that we chose to evaluate in our D&I pilot study. These outcomes were intentionally incorporated and operationalized in our interview guides.

Study procedures

We used a pre- and post-implementation pilot study design to evaluate our D&I strategies. As part of the formative phase of our research, we interviewed organizations leading this work to inform our dissemination and implementation plans. After conceptualizing and implementing these plans, we assessed their effectiveness by evaluating reach – the number of individuals engaging with the website through the D&I strategies – using self-reported survey data and Google Analytics. Our study, therefore, employed a multimethod approach, using qualitative interview data to develop and refine our strategies and quantitative data to assess their reach. We chose six user sites in one urban Midwestern city to implement the website: two primary care clinics, two school-based clinics, and two secondary schools (see Table 1). At pre-implementation, team members interviewed one to two staff members (n = 8 individuals) at these six user organizations to co-create an individualized D&I plan for each organization that detailed how they could promote the website within their respective settings. Staff members, who were recruited to complete the interviews, were largely representative of the individuals within each organization who would be implementing the website (i.e., patient-facing nursing staff, health educators, and clinicians). Zoom interviews were approximately 45 to 60 min in length, audio-recorded, and conducted with two team members. Interviews focused on identifying the website’s strengths and weaknesses, potential website modifications, and facilitators and barriers to disseminating and implementing the website within their sites using the RE-AIM framework (see Appendix A). We then used the results of these interviews to create site-specific implementation guides and plans. The CBPR team created a sample implementation guide for each implementation environment and then revised this based on site-specific pre-implementation interview data. Each site then reviewed their implementation guides and provided feedback over two drafts to the CBPR team that guided further edits relevant to their workflows and patients/student populations prior to beginning the website implementation.

Table 1 Description of the six user organizations who participated in the D&I pilot study
Full size table

We also identified and engaged staff (n = 13) at seven potential disseminator organizations. These disseminator organizations included local and regional non-profit organizations, public health agencies, and academic departments in the local medical school involved in patient care and disease prevention outreach. These pre-implementation interviews followed the same topics as outlined for user organizations while also gauging disseminator organizations’ interest, availability, and vision for potential partnership with the research team for the next phase of the dissemination.

During the implementation phase, each user organization implemented the website according to their individualized D&I plan over a period of eight weeks (April–May 2022). Our CBPR team ensured a smooth transition by conducting orientation sessions with a designated representative from each user site to familiarize them with their site-specific D&I plan. These representatives remained in close contact with our team, providing regular updates on the implementation progress and seeking assistance in addressing any obstacles. To assess the website’s reach at each user site, we employed Google Analytics and integrated a Qualtrics survey directly into the website, allowing for self-reported data and referrals to be easily identified.

Post-implementation, we conducted Zoom interviews with representatives (n = 7) from each of the six user organizations. Interviews focused on their experiences with implementation, iterative adaptations that sites made to the D&I plans and/or materials, the facilitators and barriers to implementing the website, and their desire and intentions for ongoing use of the website at their sites. These interviews captured feedback on the D&I pilot experience across the three user organization site types, identified areas for future improvement in website dissemination using the RE-AIM framework, and evaluated disseminator organizations’ availability and preferences for leadership roles in the next phases of the website dissemination (see Appendix A). We also completed post-implementation Zoom interviews with representatives (n = 8) from the seven disseminator organizations that informed next steps in our potential partnership on dissemination and strategies to further expand the website’s reach. All post-implementation interviews were approximately 45–60 min in length and audio-recorded to allow for verbatim transcription. While this study was exempt from review by the University of Minnesota Institutional Review Board, we followed all ethical standards applicable to exempt studies including informed consent, privacy, and confidentiality.

Data analysis

All interviews were transcribed by a professional transcription service and were collaboratively coded by team members using a shared spreadsheet in Google documents. Using template analysis [26], we coded the transcripts using a priori codes derived from the RE-AIM and the HPRC dissemination frameworks. Team members met in groups of two to three to review their codebooks, discuss discrepancies in coding and reach consensus on the final codes. Finally, the entire team reviewed the high-level summaries from the interview transcripts and discussed the interpretation to achieve broad consensus.

Based on this analysis, our team developed preliminary individualized D&I plans and resources for each of the six user sites. These included implementation guides, knowledge quizzes, dissemination tracking tools, posters, handouts, and a website navigation tutorial. We then met a second time with our pre-implementation interviewees at each user organization to review the drafted D&I plans and tools to allow for their feedback to guide additional revisions prior to the final materials utilized in the implementation phase.

To assess reach, we reported descriptive frequencies for two sources of quantitative user data during the implementation phase: Google Analytics and a Qualtrics survey embedded in the website. Google Analytics is a free service provided by Google that provides comprehensive statistics on website visits [27]. It is a tracking tool that captures traffic data by embedding a small HTML code on each page of the website. With Google Analytics, website owners gain insights into how visitors discovered the site and how they engage with its content. For the purpose of this D&I pilot study, we collected and reported descriptive frequencies on new users and the average duration each new user spent on the Hmong Promoting Vaccines website. We also included an optional Qualtrics survey directly on the homepage, asking users to self-report their roles (e.g., student, teacher, parent, health care provider, or other) and how they heard about the website (e.g., referral from school, school-based clinic, primary care clinic, social media, friends/family, or other). Our sample size was determined by convenience sampling, which was influenced by budgetary and timeline constraints.

Results

Pre-implementation qualitative findings

The majority of organizations acknowledged the value of the website’s content and cultural appeal and appreciated the fact that it was the first culturally- and linguistically tailored HPV educational website for Hmong adolescents and their parents. They suggested a few website modifications to increase functionality and resonance for users, including embedding a Qualtrics survey to capture self-reported referrals to the web site.

Analysis of user organizations’ pre-implementation interviews revealed that each site had unique D&I needs that depend on factors including primary stakeholders, workflow processes, decision-making processes, ease of implementation, and usefulness to the organization and its audience (see Table 2). User organizations identified several strategies to support the website’s implementation among their patients and/or students. These included selecting designated personnel responsible for delivering the website to patients/students, integrating website promotion into existing workflows, developing key talking points for engaging with them, and distributing outreach materials such as handouts and posters with QR codes. These implementation strategies were chosen with maintenance in mind, aiming to be cost-effective and minimize disruption to workflow processes within the user organization. Also, many user organizations expressed interest in tracking the reach of their implementation efforts among Hmong individuals and assessing the effectiveness of those efforts in increasing HPV knowledge and HPV vaccine uptake.

Table 2 Condensed illustration of pre-implementation interview excerpts and specific D&I plans for the six user organizations
Full size table

Pre-implementation interviews with disseminator organizations also echoed many of the findings from user organization discussions. Disseminator organizations emphasized the value of expanding reach through schools and after school programs with large Hmong student representation while also engaging parents, who are ultimately the vaccine decision makers. They shared opportunities to leverage influential voices within the Hmong community, such as trusted faith leaders and physicians. They also highlighted the challenges of identifying the best strategies to effectively disseminate an intervention that targets one cultural population within organizations that serve diverse clienteles.

Based on this feedback from both user and disseminator organizations, our team finalized the D&I plans for each user site prior to implementation (see Table 2). These plans included suggested workflows that outlined specific tasks and roles for all the implementing actors involved, such as medical assistants, clinic coordinators, clinicians, and health educators. For example, at both primary care clinic sites, it was important that clinicians were encouraged to engage families in conversation about the website. Additionally, the research team created dissemination products (e.g., handouts, posters) to support implementation activities at each user site. As suggested by all user organizations, it was important to have handouts and posters available for either parents or teens to directly access the website. We iteratively developed these plans as described in the methods. For example, at one clinic user organization, we focused on identifying potential families for intervention delivery during the rooming process and developed a typed message that could be integrated into the electronic health record handouts printed after the clinic visit. At one secondary school site, the website curriculum was incorporated into an existing unit on sexually transmitted infections, so we worked with the instructor of the course to develop a comprehensive test to assess learning and provided a list of activities to promote applied learning and sharing this information outside the classroom (see Appendix B). Each site then implemented their final individualized D&I plan.

Post-implementation quantitative findings

During the eight-week pilot period, 300 new users were captured from Google Analytics. On average, these users spent 12 min on the website, possibly facilitated by its use in the classroom. Within the Qualtrics survey, 119 users provided responses. Of the 119 users, 117 self-reported as students and two as health care providers. All 117 students shared that they were referred to the website from their health education classroom; one health care provider shared that they were referred from a community contact.

Post-implementation qualitative findings

At the post-implementation interviews, each user site described their facilitators, barriers, and plans for future dissemination (see Table 3). For the primary care and school-based clinics, the main facilitator to implementation was having a D&I plan that fit in with their usual workflow, while the main barriers were time and staff limitations. They all referred parents and adolescents to the website; none consistently opened up the website on a browser to demonstrate its use or asked patients and/or their parents to interact with the website on site. While primary care and school-based clinic staff did not know if people ever looked at the website and were uncertain about the website’s impact on HPV vaccine rates, they valued the educational resource specifically aimed at the Hmong community. For health education classrooms, the website fit smoothly into the existing curriculum; health educators reported that they perceived the students’ interest in the website and they could directly measure students’ increase in knowledge after completing the website using pre-/post-testing. The major constraints for health educators were limited time and energy to expand activities beyond the classroom and one school’s specific policies on restricting conversations and curricula around sexual health. All clinic and school sites expressed intent to continue incorporating the website into their care or education of Hmong adolescents and parents.

Table 3 Post-implementation findings by facilitators and barriers with supporting excerpts from user organizations
Full size table

Post-implementation interviews with disseminator organizations revealed several key considerations to further expand the reach of the Hmong Promoting Vaccines website. First, the framing of cancer prevention messaging is important as it could have a greater impact on vaccine-hesitant parents and avoids emphasizing a potentially stigmatizing topic like sexual activity. As one disseminator organization shared, “I know that [it] may be a culturally difficult thing to talk about or to get around but I actually don’t think we should even be talking about the shot as being attached to sexually transmitted disease. It’s a cancer prevention shot.” Second, dissemination efforts should involve trusted organizations and individuals who identify as members of the Hmong community, who can serve as sponsors or hosts and actively participate in on-the-ground dissemination efforts. Additionally, efforts should be expanded beyond the confines of a clinic or educational setting and include outreach in platforms like social media, community events involving Hmong adolescents and parents such as sport competitions or cultural fairs, and engagement with faith-based organizations. Leveraging existing connections with organizations that have sexual health curricular programs can also help reach target audiences effectively. Moreover, the scope of these efforts should be expanded beyond the current study location where there are high concentrations of Hmong residents, such as in Wisconsin and California. As one community advocate from a disseminator organization noted, “[It’s okay if your team] wants to test it [the website] out in the most dense area first but I would like to see what this would look like in smaller places. Places that don’t have so much Hmong people how are their communities reacting to this? Because they matter too. They're the ones that struggle in terms of getting the resources and getting support.”

Disseminator organizations recognized the crucial role Hmong parents play in their children’s vaccination and recommended targeted outreach to parents at community events and through school initiatives. Some organizations suggested expanding the outreach to the broader community and non-Hmong audiences. However, other community partners expressed concerns that tailored materials might be misinterpreted by non-Hmong recipients. They worried this could lead to misconceptions that HPV infections are specific to the Hmong community, undermining the message’s effectiveness and attributing HPV vaccine uptake and cervical cancer disparities to the Hmong community rather than structural determinants. To address this, disseminator organizations advised adapting the website content and messaging for a broader audience to prevent misinterpretation and improve relevance.

Our research team responded by developing a dissemination toolkit (z.umn.edu/hmonghpvtoolkit) to help organizations integrate and share the website in their local contexts, incorporating feedback from user and disseminator organizations. The toolkit includes resources suitable for various settings, offering detailed workflow guidance for healthcare and educational environments. It provides specific talking points for different contexts and a collection of dissemination materials, such as after-visit summaries, posters, and handouts, for adolescent patients, parents, and students.

Discussion

This pilot D&I study demonstrated the value of using a CBPR approach to develop and pilot test D&I strategies of our culturally-tailored Hmong Promoting Vaccines website in primary care clinics, school-based health clinics, and school health education classrooms. Through our partnerships with user and disseminator organizations, we developed and honed our D&I strategies for expanding the reach of our website in health care and educational settings and then evaluated them in these contexts. Our partnering organizations shared positive feedback about the website as a tool for supporting HPV and HPV vaccine knowledge and attitude changes with Hmong American patients and students and their parents, which aligned with our previous feasibility pilot findings with adolescent-parent dyads. However, our D&I pilot also demonstrated challenges with reach, especially in the clinical settings, and with engaging patients and students with the materials at the point of care. Throughout our project, community engagement provided essential context and collaboration to ground our dissemination efforts in strategies that would work best for our intended audience.

Dissemination strengths of our website in clinics and school classrooms included its cultural tailoring and the way in which the website leveraged an existing need for information on HPV and the HPV vaccine with resources and processes that fit within the existing workflows of the user sites. Previous interventions developed for specific cultural groups have described similar benefits of cultural tailoring to enhance the appeal and uptake of the intervention [28, 29], underscoring the value of community-engaged approaches to tailored interventions. Furthermore, our partnership with user organizations to develop individualized resources and D&I plans helped us to more closely align the website implementation with identified needs and to integrate the dissemination within their existing workflows, such as pre-visit planning to identify eligible patients in clinics and sharing handouts during the rooming process for patients and their parents to review while waiting for their clinician. This integration thereby facilitated the promotion of the website for participating schools and clinics. The ability to adapt the implementation strategies to both the stated need of the organizations and to their existing workflows are well-described facilitators to the D&I of health interventions [30, 31] and an important consideration when designing implementation strategies in D&I trials [32].

Several issues arose during this pilot study that offer opportunities to iteratively revise our website’s future D&I efforts. Major challenges included limited time and staffing to consistently promote the website and demonstrate its use to students and/or patients in real time, a commonly cited barrier to D&I of an intervention [33, 34]. High levels of staff turnover in primary care clinics further limited dissemination efforts in this context because new staff often lacked familiarity with the website and the processes for promoting it among patients and parents. Working within these contextual constraints, efforts to streamline the identification of potential intervention recipients and to seamlessly integrate resources promoting the website into existing workflows may increase the website’s reach [35]. We further noted a strong seasonality effect with well child visit timing and a preference to link the website with HPV vaccine conversation at these preventive visits that will be an important consideration for future clinic dissemination. Expanding the reach of the dissemination to the parents who are typically making vaccine-related decisions and finding ways to better match website promotion resources to the preferences of today’s adolescents and parents are two other challenges to address. Further iterative modification of dissemination strategies in conjunction with community partners, as described in the bidirectional interchange between disseminator organizations and user organizations in the HPRC framework [23] will be important to explore preferences for receiving information on the website.

Our engagement with potential dissemination partners throughout this study has enhanced our understanding of how to effectively promote the website in schools and clinics. This process also highlighted potential challenges with our D&I plans at these sites and identified opportunities to expand the website's reach within the broader community. However, feedback from disseminator organizations was occasionally in conflict and required more in-depth exploration between disseminator organizations and with our CAB members and our CBPR team to weigh the challenges and strengths of dissemination strategies. Conversations about expanding the website’s reach is a key example of a tension that arose in our pilot study findings from our post-implementation interviews with disseminators. While several disseminator organizations proposed expanding the reach out into the community and even beyond Hmong-identifying audiences, others shared concerns about how these materials might be interpreted without clear messaging to explain the focus on Hmong adolescents and parents. Just as minoritized populations may not find that dissemination materials developed for general audiences resonate with their experiences, the fit and relevance of materials developed for a specific cultural group may not translate well to broader audiences [9].

Our results support using a community-engaged and participatory approach in the dissemination and implementation of culturally tailored interventions designed to address health inequities in minoritized groups for several reasons. First, the early engagement of community members through our CBPR team and CAB contributed helpful contextual knowledge to guide the selection of user organization sites and D&I approaches for the website. We then built on this momentum of engagement in our partnerships with local and regional organizations to develop and test individualized D&I resources and strategies for each site’s unique needs and preferences. This type of early community engagement in D&I research allows for the integration of strategies that respond to the upstream social determinant assets and challenges of a particular community, and holds great promise for increasing intervention uptake and meaningful use [9]. In this way, our project highlights how CBPR approaches to implementation science are well poised to address the challenge of external validity in intervention implementation by engaging community members in the process of identifying and prioritizing modifications for their specific contexts, one of the many previously described strengths of CBPR application in implementation science research [15]. Our approach, and that of others highlighted here [17,18,19], offer a playbook for how the implementation science field might increase the speed of scaling out research to address health inequities by engaging voices on the margins who are intimately familiar with the context to determine whether interventions should be modified or remain unchanged. Using CBPR in dissemination and implementation not only enhances the cultural relevance of interventions but also creates adaptable pathways for broader implementation across diverse contexts. Furthermore, our early engagement with a range of community partners guided iterative modifications of our strategies and methods to strengthen future dissemination and implementation efforts. This approach also helped build relationships with collaborating user organizations, reducing the challenges of implementing our website in various contexts. This resulted in our ability to better enhance the alignment of our D&I approaches to user organizations’ institutional systems, which we anticipate will increase their willingness and capacity to continue partnering on dissemination and that may ultimately translate to increased sustainability [8, 15, 36]. Finally, our involvement of community organizations from both disseminator and user organization pools brought important collaboration and perspectives that can help to reduce the siloed work that often exists in these spaces and allow for more shared power, decision-making, and resources as described by Brownson and colleagues. [10]. In summary, CBPR strengthens the generalizability of implementation efforts by embedding local knowledge, fostering adaptability, and maintaining cultural and contextual relevance. It provides a robust framework for delivering interventions that address health inequities while promoting sustainability and collaboration.

Limitations

Our study had several limitations. As a pilot D&I study, it had a low number of user organizations, disseminator organizations and Hmong people who visited the website from clinics and school-based clinics. The limited sample size and conveniently sampled organizations restrict the generalizability of these findings. The organizations selected for participation may already have recognized the need to educate and promote HPV vaccination awareness within their Hmong constituencies, introducing a potential selection bias. Additionally, most of the participating organizations were local to or near the city of the partnered academic institution, which has a large resident Hmong population. These findings may primarily apply to similar settings with larger Hmong populations. Future dissemination efforts will provide opportunities for further evaluation and greater generalizability of the website. Additionally, we did not directly assess users’ experiences with the website dissemination processes, and thus may have missed important perspectives to guide future D&I efforts in these spaces. We also struggled with accurate measures of reach, both in terms of numbers of users and where they learned about the website, due to the optional nature (i.e., self-report bias) of the Qualtrics survey and the limitations of Google Analytics. In future studies, incorporating more robust measures of website engagement beyond time spent on the site would also provide useful information for how dissemination strategies could enhance the use of the website. Finally, we conducted this study in one cultural group within one geographic area, which may limit the generalizability of our findings to Hmong communities in other regions and to other communities.

Conclusion

Through this pilot D&I study, we successfully demonstrated how community engagement informed the iterative development of the dissemination and implementation of a culturally-tailored educational website tool for HPV education across various settings, including primary care clinics, school-based health clinics, and school health education classrooms. By employing the HPRC and RE-AIM frameworks, our engagement of user and disseminator community organizations provided valuable insights and guidance for our dissemination and implementation efforts for reach, effectiveness, adoption, implementation, and maintenance. Additionally, using a CBPR approach allowed us to uncover implementation insights to enhance the intervention’s contextual validity and sustainability.

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

CAB:

Community Advisory Board

CBPR:

Community-based participatory research

D&I:

Dissemination and implementation

ETI:

Educational Technology Innovations

HPRC:

Health Promotion Research Center

HPV:

Human papillomavirus

RE-AIM:

Reach, effectiveness, adoption, implementation, and maintenance

References

  1. Mills PK, Yang RC, Riordan D. Cancer incidence in the Hmong in California, 1988–2000. Cancer. 2005;104:2969–74. https://doiorg.publicaciones.saludcastillayleon.es/10.1002/cncr.21525.

    Article  PubMed  Google Scholar 

  2. Oliver SE, Unger ER, Lewis R, McDaniel D, Gargano JW, Steinau M, et al. Prevalence of Human Papillomavirus Among Females After Vaccine Introduction—National Health and Nutrition Examination Survey, United States, 2003–2014. J Infect Dis. 2017;216:594–603. https://doiorg.publicaciones.saludcastillayleon.es/10.1093/infdis/jix244.

    Article  PubMed  Google Scholar 

  3. Xiong S, Kasouaher MY, Vue B, Culhane-Pera KA, Pergament SL, Desai J, et al. “We will do whatever it takes”: Understanding Socioecological Level Influences on Hmong-American Adolescents and Parents’ Perceptions of the Human Papillomavirus Vaccine. J Cancer Educ. 2022;37:1893–901. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s13187-021-02057-4.

    Article  PubMed  Google Scholar 

  4. Lee HY, Vang S. Barriers to Cancer Screening in Hmong Americans: The Influence of Health Care Accessibility, Culture, and Cancer Literacy. J Community Health. 2010;35:302–14. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s10900-010-9228-7.

    Article  PubMed  Google Scholar 

  5. Lor M. Systematic Review: Health Promotion and Disease Prevention Among Hmong Adults in the USA. J Racial Ethn Health Disparities. 2018;5:638–61. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s40615-017-0410-9.

    Article  PubMed  Google Scholar 

  6. Lee HY, Xiong S, Sur A, Khang T, Vue B, Culhane-Pera KA, et al. Evaluating Human Papillomavirus eHealth in Hmong Adolescents to Promote Vaccinations: Pilot Feasibility Study. JMIR Form Res. 2023;7:e38388. https://doiorg.publicaciones.saludcastillayleon.es/10.2196/38388.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Alcaraz KI, Yanez BR. Interventions to promote health equity: implications for implementation science in behavioral medicine. Transl Behav Med. 2022;12:885–8. https://doiorg.publicaciones.saludcastillayleon.es/10.1093/tbm/ibac062.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Baumann AA, Cabassa LJ. Reframing implementation science to address inequities in healthcare delivery. BMC Health Serv Res. 2020;20:190. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12913-020-4975-3.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Fitzpatrick SL, Glenn BA, Duru OK, Ford CL. Health Equity in Dissemination and Implementation Science. Dissemination and Implementation Research in Health, Oxford University PressNew York; 2023, p. 534–57. https://doiorg.publicaciones.saludcastillayleon.es/10.1093/oso/9780197660690.003.0025.

  10. Brownson RC, Kumanyika SK, Kreuter MW, Haire-Joshu D. Implementation science should give higher priority to health equity. Implement Sci. 2021;16:28. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s13012-021-01097-0.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Shato T, Kepper MM, McLoughlin GM, Tabak RG, Glasgow RE, Brownson RC. Designing for dissemination among public health and clinical practitioners in the USA. J Clin Transl Sci 2024;8:e8. https://doiorg.publicaciones.saludcastillayleon.es/10.1017/cts.2023.695.

  12. McLoughlin GM, Martinez O. Dissemination and Implementation Science to Advance Health Equity: An Imperative for Systemic Change. Commonhealth (Phila). 2022;3:75–86.

    Article  PubMed  Google Scholar 

  13. Bodison SC, Sankaré I, Anaya H, Booker-Vaughns J, Miller A, Williams P, et al. Engaging the Community in the Dissemination, Implementation, and Improvement of Health-Related Research. Clin Transl Sci 2015;8:814–9. https://doiorg.publicaciones.saludcastillayleon.es/10.1111/cts.12342.

  14. Blachman-Demner DR, Wiley TRA, Chambers DA. Fostering integrated approaches to dissemination and implementation and community engaged research. Transl Behav Med. 2017;7:543–6. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s13142-017-0527-8.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Wallerstein N, Duran B. Community-Based Participatory Research Contributions to Intervention Research: The Intersection of Science and Practice to Improve Health Equity. Am J Public Health. 2010;100:S40–6. https://doiorg.publicaciones.saludcastillayleon.es/10.2105/AJPH.2009.184036.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Allicock M, Haynes-Maslow L, Johnson L-S, Carpenter WR, Vines AI, Belle DG, et al. Peer Connect for African American breast cancer survivors and caregivers: a train-the-trainer approach for peer support. Transl Behav Med. 2017;7:495–505. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s13142-017-0490-4.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Martindale-Adams J, Tah T, Finke B, LaCounte C, Higgins BJ, Nichols LO. Implementation of the REACH model of dementia caregiver support in American Indian and Alaska Native communities. Transl Behav Med. 2017;7:427–34. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s13142-017-0505-1.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Kwon SC, Patel S, Choy C, Zanowiak J, Rideout C, Yi S, et al. Implementing health promotion activities using community-engaged approaches in Asian American faith-based organizations in New York City and New Jersey. Transl Behav Med. 2017;7:444–66. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s13142-017-0506-0.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Gore R, Patel S, Choy C, Taher M, Garcia-Dia MJ, Singh H, et al. Influence of organizational and social contexts on the implementation of culturally adapted hypertension control programs in Asian American-serving grocery stores, restaurants, and faith-based community sites: a qualitative study. Transl Behav Med. 2020;10:1525–37. https://doiorg.publicaciones.saludcastillayleon.es/10.1093/tbm/ibz106.

    Article  PubMed  Google Scholar 

  20. Villalobos A, Blachman-Demner D, Percy-Laurry A, Belis D, Bhattacharya M. Community and partner engagement in dissemination and implementation research at the National Institutes of Health: an analysis of recently funded studies and opportunities to advance the field. Implement Sci Commun. 2023;4:77. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s43058-023-00462-y.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Northridge ME, Kavathe R, Zanowiak J, Wyatt L, Singh H, Islam N. Implementation and dissemination of the Sikh American Families Oral Health Promotion Program. Transl Behav Med. 2017;7:435–43. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s13142-017-0466-4.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Dhaliwal BK, Seth R, Thankachen B, Qaiyum Y, Closser S, Best T, et al. Leading from the frontlines: community-oriented approaches for strengthening vaccine delivery and acceptance. BMC Proc. 2023;17:5. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12919-023-00259-w.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Harris JR, Cheadle A, Hannon PA, Forehand M, Lichiello P, Mahoney E, et al. A framework for disseminating evidence-based health promotion practices. Prev Chronic Dis. 2012;9:E22.

    PubMed  Google Scholar 

  24. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89:1322–7. https://doiorg.publicaciones.saludcastillayleon.es/10.2105/AJPH.89.9.1322.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Gaglio B, Shoup JA, Glasgow RE. The RE-AIM Framework: A Systematic Review of Use Over Time. Am J Public Health. 2013;103:e38-46. https://doiorg.publicaciones.saludcastillayleon.es/10.2105/AJPH.2013.301299.

    Article  PubMed  PubMed Central  Google Scholar 

  26. King N. Template analysis. Qualitative methods and analysis in organizational research: A practical guide., Thousand Oaks, CA: Sage Publications Ltd; 1998, p. 118–34.

  27. Short CE, DeSmet A, Woods C, Williams SL, Maher C, Middelweerd A, et al. Measuring Engagement in eHealth and mHealth Behavior Change Interventions: Viewpoint of Methodologies. J Med Internet Res. 2018;20:e292. https://doiorg.publicaciones.saludcastillayleon.es/10.2196/jmir.9397.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Griffith DM, Efird CR, Baskin ML, Webb Hooper M, Davis RE, Resnicow K. Cultural Sensitivity and Cultural Tailoring: Lessons Learned and Refinements After Two Decades of Incorporating Culture in Health Communication Research. Annu Rev Public Health. 2024;45:195–212. https://doiorg.publicaciones.saludcastillayleon.es/10.1146/annurev-publhealth-060722-031158.

    Article  PubMed  Google Scholar 

  29. Joo JY, Liu MF. Culturally tailored interventions for ethnic minorities: A scoping review. Nurs Open. 2021;8:2078–90. https://doiorg.publicaciones.saludcastillayleon.es/10.1002/nop2.733.

    Article  PubMed  Google Scholar 

  30. Stirman SW, Baumann AA, Miller CJ. The FRAME: An expanded framework for reporting adaptations and modifications to evidence-based interventions. Implementation Science 2019;14. https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s13012-019-0898-y.

  31. Cook CR, Lyon AR, Locke J, Waltz T, Powell BJ. Adapting a Compilation of Implementation Strategies to Advance School-Based Implementation Research and Practice. Prev Sci. 2019;20:914–35. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s11121-019-01017-1.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Kilbourne A, Chinman M, Rogal S, Almirall D. Adaptive Designs in Implementation Science and Practice: Their Promise and the Need for Greater Understanding and Improved Communication. Annu Rev Public Health. 2024;45:69–88. https://doiorg.publicaciones.saludcastillayleon.es/10.1146/annurev-publhealth-060222-014438.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Brown EJ, Kangovi S, Sha C, Johnson S, Chanton C, Carter T, et al. Exploring the Patient and Staff Experience With the Process of Primary Care. The Annals of Family Medicine. 2015;13:347–53. https://doiorg.publicaciones.saludcastillayleon.es/10.1370/afm.1808.

    Article  PubMed  Google Scholar 

  34. Nieuwland A, O’Mara JR, Mackenbach JD. Exploring public health researchers’ approaches, barriers, and needs regarding dissemination: A mixed-methods exploration. J Clin Transl Sci. 2024;8:e81. https://doiorg.publicaciones.saludcastillayleon.es/10.1017/cts.2024.527.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Staras S, Tauscher JS, Rich N, Samarah E, Thompson LA, Vinson MM, et al. Using a Clinical Workflow Analysis to Enhance eHealth Implementation Planning: Tutorial and Case Study. JMIR Mhealth Uhealth. 2021;9:e18534. https://doiorg.publicaciones.saludcastillayleon.es/10.2196/18534.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Boothroyd RI, Flint AY, Lapiz AM, Lyons S, Jarboe KL, Aldridge WA II. Active involved community partnerships: co-creating implementation infrastructure for getting to and sustaining social impact. Transl Behav Med. 2017;7:467–77. https://doiorg.publicaciones.saludcastillayleon.es/10.1007/s13142-017-0503-3.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the American Cancer Society – MN Chapter, Community-University Health Care Center, Hmong American Center, Hmong College Prep Academy, Minnesota Community Care, Minnesota Department of Health, Phalen Village Clinic, Planned Parenthood North Central States, University of Minnesota Masonic Cancer Center, and Washington Technology Magnet School, for their contributions to this research project.

Funding

Research reported was supported by the University of Minnesota Program in Health Disparities Research (PHDR) Health Disparities Pilot Grant award HD21MCC01. This award is administrated by the UMN PHDR and funded by the Masonic Cancer Center, University of Minnesota, #NIH P30 CA77598. The content is solely the responsibility of the authors and does not necessarily represent the official views of the University of Minnesota.

Author information

Authors and Affiliations

  1. Department of Family Medicine and Community Health, University of Minnesota Medical School, 717 Delaware St SE, Minneapolis, MN, 55414, United States

    Serena Xiong, Kathleen A. Culhane-Pera & April K. Wilhelm

  2. Somali, Latino, and Hmong Partnership for Health and Wellness (SoLaHmo), Community-University Health Care Center (CUHCC), 2001 Bloomington Ave, Minneapolis, MN, 55404, United States

    Serena Xiong, Kathleen A. Culhane-Pera, Tounhia Khang, Maria Beatriz Torres & Bai Vue

  3. Chronic Disease and Environmental Epidemiology, Minnesota Department of Health (MDH), PO Box 64975, St. Paul, MN, 55164, United States

    Jay Desai

Authors
  1. Serena Xiong
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Kathleen A. Culhane-Pera
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jay Desai
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Tounhia Khang
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Maria Beatriz Torres
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Bai Vue
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. April K. Wilhelm
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

SX, JD, and AW conceived the idea for the study and oversaw the research. SX, TK, KCP, MBT, BV, and AW constructed and refined the study protocol. SX, TK, BV, and AW conducted, acquired, and managed the data. SX, TK, KCP, MBT, BV, and AW were involved in the data collection and analysis. SX and AW drafted, refined, and revised the manuscript. All authors, edited, read, and approved the final manuscript.

Corresponding author

Correspondence to Serena Xiong.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiong, S., Culhane-Pera, K.A., Desai, J. et al. Hmong Promoting Vaccines eHealth website: a community-based participatory research pilot to evaluate dissemination and implementation strategies for primary care and educational contexts. Implement Sci Commun 6, 45 (2025). https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s43058-025-00733-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s43058-025-00733-w

Keywords

Implementation Science Communications

ISSN: 2662-2211