- Study protocol
- Open access
- Published:
Development and evaluation of an implementation strategy to increase HPV vaccination among underserved youth across Texas: a protocol paper
Implementation Science Communications volume 5, Article number: 143 (2024)
Abstract
Background
All for Them is a theory-based and evidence-informed multilevel, multicomponent program delivered through schools to increase HPV vaccination among medically underserved youth across Texas. Given the potential logistical challenges of program implementation, understanding how to best support the implementation and sustainment of the program is critical. The overall goals of this study are twofold: 1) develop a multifaceted implementation strategy, Implementing All for Them (IM-AFT); and 2) evaluate the impact of IM-AFT on implementation outcomes for schools and healthcare providers to successfully implement All for Them in their respective settings.
Methods
This study is underpinned by a comprehensive and synergistic conceptual framework. We will use Social Cognitive Theory and the Interactive Systems Framework for Dissemination and Implementation to inform important individual and organizational factors to target as part of implementation strategy development. We will use this formative work and Implementation Mapping, a systematic, iterative process that guides the use of theories, models, and frameworks, to develop IM-AFT. The three core aims of the present study are connected to the five tasks of Implementation Mapping. For Aim 1, we will develop IM-AFT using a community-based participatory research-informed approach, including a qualitative assessment of needs and assets associated with program implementation and identification of behavioral and psychosocial objectives to determine implementation outcomes. For Aim 2, we will use a mixed-methods approach to assess user experience with the IM-AFT prototype to test its feasibility, usability, and acceptability. For Aim 3, we will use a descriptive checklist to assess the impact of IM-AFT on user fidelity of program implementation.
Discussion
This paper presents the detailed protocol for developing and evaluating IM-AFT to successfully implement All for Them, leveraging a systematic, community- and theory-based approach and user experience with the strategy prototype. This study will contribute to expanding limited scientific knowledge about using multiple sources to develop and evaluate specified implementation strategies for effective implementation of school-based vaccination programs. Theory-based IM-AFT will guide collaborations between schools and community health centers to improve HPV and other adolescent vaccination rates in underserved communities in Texas.
Background
Adolescent HPV vaccine uptake remains suboptimal in the United States (US) with geographic disparities in vaccine mandates and coverage. Nonetheless, there is a positive trend in recent HPV vaccine series completion rates among US adolescents ages 13–17 years, which rose from 51% in 2018, to 58.6% in 2020, and to 61.4% in 2023 [1]. Efforts that contribute to this increase include multilevel interventions for HPV vaccine promotion delivered in nonprimary care settings, including schools [2,3,4]. Research has shown systems (e.g., health clinics) and individuals (e.g., providers and parents) as the most influential levels when developing multilevel interventions [5].
All for Them is a theory-based and evidence-informed multilevel, multicomponent school-based program designed to increase HPV vaccination among medically underserved youth across Texas, a state with one of the lowest HPV vaccination completion rates (58.5%) and no HPV vaccine mandate [6]. All for Them comprises three evidence-based synergistic strategies: 1) a parent-focused social marketing and educational campaign to promote positive HPV vaccine attitudes, 2) comprehensive school-based vaccination clinics, and 3) HPV-focused continued nursing education for school nurses including best communication practices about HPV vaccination. Since 2017, All for Them has provided comprehensive immunization services and HPV education to a total of 76 middle schools and 13 high schools in six Texas school districts in collaboration with five vaccination providers.
While All for Them is a promising approach, program implementation can be challenging as it requires collaboration between schools and providers. For example, competing priorities in schools and clinic staff shortages may hinder coordination of vaccine clinic events between providers and schools. In addition, school nurses may be apprehensive about and resist implementation due to anticipated added burden of work (e.g., distribution and collection of consent forms, clinic coordination activities). These logistical challenges can be mitigated by developing and selecting implementation strategies, which are methods or techniques used to enhance adoption, implementation, sustainability, and scale-up of interventions [7]. Implementation strategies can involve a single component (i.e., discrete strategy) or multiple components (i.e., multifaceted strategy) [8]. To develop multifaceted implementation strategies, authorities recommend using systematic and iterative approaches. One such approach is Implementation Mapping, a step-by-step process that guides the use of theories, models, frameworks, empirical evidence, and partner input to develop and select implementation strategies [9].
Understanding how to best support the implementation and sustainment of All for Them is critical given the program’s potential to bolster schools’ and healthcare providers’ (e.g., community health centers, county health departments, immunization-focused community organizations) commitment to prioritizing and increasing adolescent HPV vaccination, thereby ultimately reducing HPV-related cancers. The overall goals of this study are to: 1) develop a multifaceted implementation strategy – Implementing All for Them (IM-AFT) – using Implementation Mapping, and 2) evaluate the impact of the strategy on implementation outcomes for schools and community health centers to successfully implement All for Them in their respective settings. The specific aims of this study are to: 1) establish advisory partnerships and develop IM-AFT using a community-based participatory research-informed approach; 2) assess feasibility, acceptability, and usability of IM-AFT with schools and community health centers; and 3) assess the impact of IM-AFT on the implementation fidelity of All for Them with the participating schools and community health centers. This protocol describes the process for conducting each of the aims. Reporting of this protocol adheres to the Standards for Reporting Implementation Studies (StaRI) [10]; the StaRI checklist is included as supplementary material (Supplementary Material 1).
Methods
Theoretical model
This study is underpinned by a comprehensive and synergistic conceptual framework (Fig. 1). We draw from the Interactive Systems Framework for Dissemination and Implementation (ISF), which explains the process by which health programs are introduced into a setting and posits that organizational readiness plays an essential role [11]. Within the ISF, readiness (R) is a combination of motivation (M) x innovation-specific capacity (C) x general capacity (C), abbreviated as R = MC2 [12]. Within each readiness component, subcomponents form an organization’s overall readiness (e.g., climate, priority, staff knowledge and skills). We use the ISF and R = MC2 heuristic to understand important organizational barriers and facilitators to inform implementation strategy development. We also use Social Cognitive Theory [13] to specify individual determinants (e.g., knowledge, self-efficacy, outcome expectations) of implementation behaviors. Subsequently, we use Implementation Mapping to design and develop implementation strategies. To evaluate the implementation strategies, we use the Implementation Outcomes Framework [14] to specify relevant outcomes. As Implementation Mapping is the centerpiece of our conceptual framework, the three core aims of the study are connected to its sequence of five tasks (Table 1). Aim 1 uses Implementation Mapping Tasks 1–4, while Aims 2 and 3 are conducted as part of Task 5.
-
Aim 1. Establish advisory partnerships and develop the Implementing All for Them (IM-AFT) multifaceted implementation strategy using a community-based participatory research-informed approach.
As an initial step to developing IM-AFT, we will establish an advisory group (All for Them Advisory Board – Dissemination Subcommittee) comprising members from school districts, healthcare providers, and experts in HPV content, research quality improvement, dissemination, and community engagement. This group will collaborate with the research team throughout the project to provide insights about the development and evaluation of the implementation strategy using a community-based participatory research-informed approach. Community-based participatory research is a collaborative approach that equitably involves all partners in all aspects of the research process and recognizes the unique strengths that each brings [15]. Shared knowledge and experiences can result in more relevant and effective interventions [15]. We will be leveraging the advisory groups members’ input and conducting key informant interviews with them for additional guidance on strategy development.
For Task 1, we will conduct a needs, assets, and readiness assessment to examine multilevel factors associated with All for Them implementation. This information will be collected via qualitative interviews with 10 school and provider partners who previously implemented All for Them and 10 potential partners who have not, but might, bring All for Them into their community. We will use the R = MC2 heuristic to explore school-level barriers and facilitators to All for Them implementation and adoption and use Social Cognitive Theory to explore individual-level determinants. We will use Rapid Assessment Procedures, which balance research rigor, timeliness, and utility of the findings [16]. Analysis of qualitative data will occur in three steps: 1) summarize each interview using a standardized template organized by interview question; 2) create a matrix of responses organized by respondent role and interview questions; 3) synthesize summaries of key constructs and identify gaps in information across respondent roles. Data analysis in tandem with data collection will allow an efficient assessment of data quality and further the development of the implementation strategy. The qualitative results, coupled with literature review and advisory group input will allow us to understand implementation drivers and inform the Implementation Mapping process.
For Task 2, we will work with the advisory group to determine implementation outcomes and performance objectives (i.e., behaviors required to achieve implementation outcomes). Performance objectives help define what certain key personnel should do to implement the program. Subsequently, leveraging information obtained from Task 1, we will identify and connect relevant determinants influencing the performance objectives. These determinants will be central to illustrate why and how one would decide to adopt and implement All for Them as intended. Finally, we will use the identified performance objectives and determinants to develop matrices of change objectives, which specify changes required in each determinant to carry out the corresponding performance objective.
Task 3 entails selecting theoretical change methods and designing implementation strategies that operationalize these methods using the matrices of change objectives constructed in Task 2. Theoretical change methods (e.g., modeling, guided practice, verbal persuasion, leadership engagement) are techniques to specifically impact implementers’ individual determinants (e.g., knowledge, attitudes, skills and self-efficacy, perceived norms) and organizational factors (e.g., leadership support) of implementation and ultimately influence the behaviors of program adopters and implementers.
Task 4 involves developing implementation protocols and materials by creating design documents and drafting content of the implementation strategies. We anticipate that IM-AFT will include multiple components such as: 1) a web-based toolkit to facilitate uptake of All for Them implementation strategies, 2) synchronous and asynchronous training for All for Them orientation and toolkit utilization guidance, and 3) technical assistance in the form of implementation learning community support sessions. Development of the protocols and materials will be followed by content refinement and rigorous pretesting by project staff (alpha test) and advisory group members (beta test). The final content and design of IM-AFT will depend on the iterative Implementation Mapping process and feedback from the advisory group. The last task of Implementation Mapping (Task 5) is implementation outcome evaluation, which is the focus of Aims 2 and 3.
-
Aim 2. Assess feasibility, acceptability, and usability of IM-AFT with schools and community health centers.
-
Aim 3. Assess the impact of IM-AFT on the implementation fidelity of All for Them with the participating schools and community health centers.
Study design
We will conduct a pilot study with potential users to evaluate IM-AFT. We will use a mixed-methods approach to assess user experience with the IM-AFT prototype to test its feasibility, usability, and acceptability (Aim 2), which will inform whether modification of IM-AFT is indicated prior to further testing [17] and dissemination. We will also assess the impact of IM-AFT on fidelity of All for Them implementation among key user types during the implementation phase (Aim 3). Pilot users in every recruited site will use all IM-AFT strategies over a period of two months immediately following strategy development. During the next phase, the sites will implement the All for Them program in their respective settings over a period of four months, during which we will collect data for Aims 2 and 3.
Setting and participants
We will recruit five sites from each user type (i.e., schools/districts and community health centers), as well as participants within each site, to participate in a 6-month implementation pilot. Leaders within each site will help identify individual potential participants. The same sample will be used for both Aims 2 and 3.
Recruitment
We will recruit participants diverse in 1) geographical type (e.g., rural, urban) and location within Texas, 2) type of lead implementer role, and 3) community size. To assist with pilot recruitment, state and national cancer-focused and healthcare association organization partners will disseminate information about the project to their clinic and school nurse contacts and members. In addition to assistance with recruitment, these partner organizations will provide feedback and expertise regarding IM-AFT development. The advisory group members will also assist with recruiting pilot sites by connecting the study team to their network.
Data collection and outcome measures
The Implementation Outcomes Framework will guide the evaluation of the pilot study outcomes. Primary outcomes for Aim 2 will be feasibility, usability, and acceptability of IM-AFT. We will assess feasibility and usability following pilot participants’ completion of the IM-AFT components (e.g., implementation support sessions) but prior to official All for Them implementation in their respective communities using electronic surveys. We will assess acceptability after All for Them implementation via semi-structured key informant interviews. In assessing feasibility, we will investigate how using IM-AFT impacts the ease with which All for Them can be implemented by users and their partners, whether school or provider, given the organizations’ existing resources and circumstances without outside intervention. To measure feasibility, we will adapt the Structured Assessment of Feasibility and a features checklist. To measure usability, we will adapt the System Usability Scale to assess users’ experiences with IM-AFT. While surveys provide valuable information, we will also conduct key informant in-depth interviews with each pilot participant to gain deeper insight into individuals’ acceptability of IM-AFT at the second phase of evaluation. Interview questions will cover satisfaction with and perceived appropriateness of program content delivery, alignment with organizational goals, barriers and facilitators, implementation of All for Them after using IM-AFT, outstanding needs, and areas for improvement. Interviews will last 45–60 min and will be audio-recorded and transcribed verbatim for coding purposes. The primary outcome for Aim 3 will be fidelity of All for Them program implementation. For data collection, we will use a fidelity checklist and implementation log that pilot participants will complete during the All for Them implementation phase. This checklist will include specific criteria to meet for the three All for Them components (social marketing and educational campaign, school-based vaccination clinics, and nursing education). Table 2 presents an overview of study outcomes and their definitions, the method of assessment, and the data collection time points.
Data analysis
For Aim 2, we will use an array of statistical analyses including descriptive statistics, correlations, measures of central tendency, and t-tests with statistical significance set at p < 0.05 to analyze the survey data. We will analyze the survey data immediately after completion of follow-up surveys and use the results to make necessary amendments to the interview guide. We will analyze the qualitative data using Rapid Assessment Procedures as described above. Given the short timeframe of the pilot implementation period, Rapid Assessment Procedures will enable us to quickly identify actionable modifications to IM-AFT. This approach will allow us to investigate anomalous or unexpected results for further exploration. We will collect and analyze quantitative and qualitative data simultaneously and compare these results for the purpose of complementarity. For Aim 3, we will develop and descriptively analyze the fidelity checklist. Descriptive analysis of the checklist will entail a preliminary understanding of what it takes for users to adhere to program implementation as intended.
Discussion
There is an urgent need for transparent, rigorous, and thorough reporting of strategies used to implement complex health interventions [10]. Implementation strategies are of paramount importance in implementation science because they constitute the ‘how to’ component of realizing the benefits of evidence-based care and producing a positive shift in healthcare practice [7]. As with all intervention research, implementation strategies need to be thoroughly and precisely delineated to enable measurement and reproducibility of their components [20]. Ideally, descriptions of implementation strategies should be ‘packaged’ in protocols or manuals describing how a given intervention is to be operationalized. This paper presents the detailed protocol for developing and evaluating a multifaceted implementation strategy, IM-AFT, to effectively implement All for Them, a multilevel, multicomponent evidence-informed program to increase HPV vaccination among underserved Texas youth. Our comprehensive conceptual framework linking relevant theories and frameworks will help identify 1) key facilitators and barriers to All for Them implementation, and 2) theoretically-informed ways to address these facilitators and barriers through change methods that are operationalized by implementation strategy components. The guiding framework will also allow pragmatic assessments of the feasibility, usability, and acceptability of IM-AFT, as well as users’ program implementation fidelity following their experience with IM-AFT. Assessing these outcomes is critical to identify implementation barriers in the specific context, which will allow for problem-solving and the identification of concrete solutions that can be implemented. These evaluations are also important to maximize the sustainability of the strategy and set the stage for more rigorous testing that is necessary to build toward larger-scale work.
Several limitations need to be considered. Implementation is rarely a sequential linear process but is rather iterative and recursive, often involving the revisiting of previous stages of exploration and pre-implementation [21]. Developing concrete implementation strategies requires considerable time and team-based effort. Therefore, employing implementation strategies as intended in a timely manner may not always be achievable in actual school and clinic settings. In addition, given the real-world nature of this work, highly controlled evaluation efforts are not feasible at this stage. Thus, the study focuses on understanding key aspects of IM-AFT and tracking how IM-AFT supports program adoption and implementation. We will gain a better understanding of how feasible, usable, and acceptable IM-AFT is to potential users given the challenges of bringing schools and providers together. As each user system has their own primary priorities, commitment to program implementation and use of implementation strategies may wane over time, affecting overall implementation fidelity and sustainability. Lastly, users’ self-report of program fidelity can introduce positive-response bias from overestimation of their adherence to implementation procedures.
Despite the limitations above, this study presents important strengths and opportunities in the intersection of implementation science and public health. First, we use a systematic, theory-based approach to develop IM-AFT that can be tested in a real-world setting and can help expand the use of All for Them at this earlier stage of the work. The strategy will have multiple components to support schools and community health centers, including building a learning community to help users overcome challenges through connection with other implementers. Thus, we will employ multiple mechanisms of action that collectively have the potential to apply across contexts. We also propose a clear rationale of how the selected theories operate in the development of IM-AFT. This process facilitates evaluation of the mechanisms of change, which will help identify essential elements and areas for improvement. Second, we will continue to learn from user experience with the IM-AFT prototype and modify the strategy as needed. The implementation support strategies are likely to be relevant, and implementation is likely to be more successful, if implementers themselves deem the implementation strategies useful and are directly involved in strategy development [22]. Finally, we assemble an advisory group comprising invested partners and audiences as well as relevant content experts who engage in the development and evaluation of IM-AFT to optimize program implementation. This study will contribute to increasing limited scientific knowledge about leveraging multiple sources to develop and evaluate highly specified implementation strategies for effective implementation of school-based vaccination programs. Theory-based IM-AFT with content expert and user input will offer an opportunity to foster collaborations between schools and community health centers to improve HPV and other adolescent vaccination rates in underserved communities in Texas.
Data availability
Not applicable.
References
Pingali C. National vaccination coverage among adolescents aged 13–17 years—national immunization survey-teen, United States, 2023. MMWR Morb Mortal Wkly Rep. 2024;73:708–14.
Escoffery C, Petagna C, Agnone C, Perez S, Saber LB, Ryan G, et al. A systematic review of interventions to promote HPV vaccination globally. BMC Public Health. 2023;23(1):1262.
Dang JH, McClure S, Gori AC, Martens T, Mojadedi A, Smith U, et al. Implementation and evaluation of a multilevel intervention to increase uptake of the human papillomavirus vaccine among rural adolescents. J Rural Health. 2023;39(1):136–41.
Rodriguez AM, Do TQN, Eyada MF, Chen L, Schmeler KM, Montealegre JR. Human papillomavirus vaccination uptake in the Rio Grande Valley: results from a pilot community-based educational and school-based vaccination program and its expansion. Vaccines. 2023;11(2):329.
Katz ML, Paskett ED. The process of engaging members from two underserved populations in the development of interventions to promote the uptake of the HPV vaccine. Health Promot Pract. 2015;16(3):443–53.
American Cancer Society. HPV Landscape Dashboard. Available from: https://www.cancer.org/cancer/risk-prevention/hpv/hpv-vaccine/hpv-texas.html. Accessed 18 Sept 2024.
Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8:1–11.
Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017;44:177–94.
Fernandez ME, Ruiter RA, Markham CM, Kok G. Intervention mapping: theory-and evidence-based health promotion program planning: perspective and examples. Front Public Health. 2019;7:209.
Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for reporting implementation studies (StaRI) statement. BMJ. 2017;356:i6795.
Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, Blachman M, Dunville R, Saul J. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. Am J Community Psychol. 2008;41:171–81.
Scaccia JP, Cook BS, Lamont A, Wandersman A, Castellow J, Katz J, Beidas RS. A practical implementation science heuristic for organizational readiness: R= MC2. J Community Psychol. 2015;43(4):484–501.
Bandura A. Social cognitive theory: an agentic perspective. Annu Rev Psychol. 2001;52(1):1–26.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38:65–76.
Winterbauer NL, Bekemeier B, VanRaemdonck L, Hoover AG. Applying community-based participatory research partnership principles to public health practice-based research networks. SAGE Open. 2016;6(4):2158244016679211.
Hamilton-Burgess C, Berkley-Patton J, Allsworth J, Bowe Thompson C, Thompson FE, Burgin T, et al. The importance of community-based and community-partnered COVID-19 testing for reducing disparities among African American populations. Health Equity. 2024;8(1):147–56.
Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. Am J Prev Med. 2009;36(5):452–7.
Lyon AR, Munson SA, Renn BN, Atkins DC, Pullmann MD, Friedman E, et al. Use of human-centered design to improve implementation of evidence-based psychotherapies in low-resource communities: protocol for studies applying a framework to assess usability. JMIR Res Prot. 2019;8(10):e14990.
Nelson MC, Cordray DS, Hulleman CS, Darrow CL, Sommer EC. A procedure for assessing intervention fidelity in experiments testing educational and behavioral interventions. J Behav Health Serv Res. 2012;39:374–96.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:1–6.
Nadeem E, Saldana L, Chapman J, Schaper H. A mixed methods study of the stages of implementation for an evidence-based trauma intervention in schools. Behav Ther. 2018;49(4):509–24.
van der Giessen JA, Ausems MG, van den Muijsenbergh ME, van Dulmen S, Fransen MP. Systematic development of a training program for healthcare professionals to improve communication about breast cancer genetic counseling with low health literate patients. Fam Cancer. 2020;19(4):281–90.
Funding
This study received funding from the Cancer Prevention and Research Institute of Texas.
Author information
Authors and Affiliations
Contributions
PMC and JC drafted the first version of the manuscript and PMC, JC, LSS, EKG, MAM, AME, and TJW substantively revised it. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Ethics approval was obtained from the Committee for the Protection of Human Subjects at the University of Texas Health Science Center at Houston (HSC-SPH-23–0302). In line with the ethics approval, informed consent will be obtained from all study participants.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Cuccaro, P.M., Choi, J., Savas, L.S. et al. Development and evaluation of an implementation strategy to increase HPV vaccination among underserved youth across Texas: a protocol paper. Implement Sci Commun 5, 143 (2024). https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s43058-024-00688-4
Received:
Accepted:
Published:
DOI: https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s43058-024-00688-4