Leveraging group model building to operationalize implementation strategies across implementation phases: an exemplar related to child maltreatment intervention selection

Abstract

Background

Implementation strategies can help support the adoption and implementation of health interventions that are appropriate for a local context and acceptable to decision makers and community members. Implementation strategies should be designed to handle the complexity of the multi-level, dynamic contexts in which interventions are implemented. Systems science theories and methods explicitly attend to complexity and can be valuable for specifying implementation strategies. Group Model Building (GMB) combines research partner engagement strategies with systems science to support researchers’ and partners’ learning about complex problems and to identify solutions through consensus. This paper specifies how GMB can operationalize implementation strategies — methods for supporting evidence implementation in real-world practice — and describes how GMB can aid in selecting and tailoring both health interventions and implementation strategies. A case study in child maltreatment prevention planning is provided to illustrate how GMB was used to specify the “actions” — strategy activities — for three implementation strategies (conduct local consensus discussions; build a coalition; model and simulate change) during the earliest implementation phases, with the goal of supporting intervention selection decisions. Examples are provided of generalizable research products that can be produced concurrently through GMB, in addition to contextually-driven implementation support.

Methods

Participants (n = 8) were engaged over four sessions using tailored GMB activities. Participants generated a qualitative system dynamics model that described their theory of change for how to prevent child maltreatment in their communities. This theory of change reflected a dynamic understanding of the interconnected determinants of child maltreatment.

Results

GMB was acceptable to participants and resulted in products that could be used for implementation planning (e.g., to model and simulate change) and future research. GMB fostered trust and idea sharing between participants.

Conclusion

GMB can facilitate learning about which outcomes are (or are not) impacted by interventions, which resources and approaches are required for quality implementation (e.g., implementation strategies), and tradeoffs in outcomes and resources between interventions. GMB also provides a structured, effective process to generate a shared implementation vision amongst participants. Lessons learned include methods for developing trust with and between participants, and the need for researchers to tailor GMB actions for participant and project needs.

Contributions to the literature

• Group model building (GMB) is a promising method to support implementation activities throughout implementation phases, and to concurrently develop research products that contribute generalizable knowledge.

• GMB provides a systematic, transparent method to operationalize strategies such as conducting local consensus discussions, modeling and simulating change, and building a coalition.

• Engaged systems science methods such as GMB can develop strong research-practice partnerships and dynamic hypotheses of change, resulting in implementation plans that are more likely to achieve desired health impacts.

• Specifying complex, multi-faceted implementation strategies can highlight common threads amongst strategy goals to identify strategy mechanisms and increase implementation efficiency.

Background

Substantial knowledge has been generated about implementation determinants (i.e., barriers and facilitators) [1] and strategies [2, 3]; however, substantial challenges remain for understanding the complex relationships between implementation determinants and health outcomes to design implementation strategies [4, 5]. Rigorous participatory methods — particularly those that leverage systems thinking to account for complex determinant relationships — can provide insights critical to understanding health and social care quality gaps, identifying potential interventions and implementation strategies to close those gaps, and anticipating intervention and strategy impacts on implementation and health outcomes over time [6].

Group model building (GMB) uses systems science thinking and methods for engaging implementation actors and other community partners to foster shared understanding about problems, generate innovative solutions, and implement solutions [7]. It has been identified as a structured approach for selecting and tailoring implementation strategies given the system’s complexity (i.e., implementation context) [4] and for testing implementation strategy mechanisms [8, 9]. More fundamentally, GMB can be leveraged as an implementation approach — a structured method for reproducibly operationalizing implementation strategies and implementation processes (e.g., intervention adoption). GMB is particularly well-suited for operationalizing strategies that target collaboration and decision-making — phenomena in each implementation phase.

This paper will: a) briefly overview GMB; b) describe how implementation strategy dimensions can be operationalized with GMB; c) detail a case study wherein GMB was used to operationalize three common implementation strategies (conduct local consensus discussions, model and simulate change, and build a coalition) to support intervention selection; and d) propose next steps for testing GMB as an implementation approach across implementation phases.

GMB: Brief overview

GMB was developed to increase consulting clients’ acceptability of system dynamics models and implementation of modeling insights [7, 10]. It has been conducted with diverse groups of community members, clinicians, and healthcare leaders [11].

GMB primarily employs system dynamics [12], although recent studies applied other systems-based modeling methods such as agent-based modeling [13]. A distinguishing feature of system dynamics is its focus on characterizing feedback dynamics that drive behaviors over time (e.g., non-linear effects, delays, accumulations). Understanding these dynamics is critical for identifying the most effective intervention targets and anticipating implementation outcomes. GMB combines qualitative and/or quantitative modeling with semi-structured group activities or “scripts.” GMB aims align well with addressing critical implementation challenges. For example, without shared understanding of a problem’s underlying dynamics, it is difficult to reach consensus on which intervention(s) are best suited to affect that problem [14,15,16]. Moreover, system dynamics models developed or validated through GMB can reduce the selection and implementation of “fixes that fail” [17] by accurately characterizing the system’s dynamics and allowing users to test assumptions about potential innovation and implementation strategy impacts. GMB and fields such as human-centered design [18] have shown that including end-users in model development is key to model validity and uptake.

GMB as an implementation approach

This section describes how GMB can operationalize implementation strategies across seven key dimensions outlined by Proctor, Powell, and McMillen [19]: actors, actions, action targets, dose, temporality, implementation outcomes, and justification (Table 1).

Table 1 Overview of How Group Model Building (GMB) Can Be Specified as an Implementation Strategy

Actors

Actors deliver implementation strategies [19]. GMB involves a core modeling team and facilitation team. One individual can fill multiple roles.

The facilitation team delivers GMB sessions. They promote curiosity and dialogue between participants by audibly synthesizing participants’ expertise and insights [63], and by pointing out dynamic phenomena as participants describe complex problems and potential solutions. Facilitators ensure that logistic needs are met (e.g., room set-up, agenda setting), engage partners, and clarify scientific and practical objectives. The facilitation team often includes 3–5 individuals, including, at a minimum, someone trained in systems thinking and/or system dynamics (“modeler facilitator”) [64]. It is helpful for a facilitation team member to have substantive expertise about the modeled problem(s) to elicit relevant insights from participants, and to help the modeling team incorporate participants' insights and relevant scientific evidence during model development. Hovmand’s Reflector Feedback script encourages the facilitator to reflect on how participants’ comments align with scientific evidence during GMB session closure [21].

To mitigate power imbalances, it is also recommended to include a “community facilitator”— someone who is part of the community from which GMB participants are drawn or who has an existing relationship with the community [64]. Community members might co-facilitate GMB sessions and/or be involved in session planning activities such as selecting and tailoring scripts to community norms, experiences, and priorities, or selecting locations that foster psychological comfort for participants. Table 1 provides guidance on identifying community facilitators and GMB participants.

The modeling team is responsible for GMB session design, often in partnership with the facilitation team [29, 64]. The modeling team includes at least one person trained in system dynamics (or the selected modeling approach) to translate participants’ insights into qualitative and/or quantitative system dynamics models and foster systems thinking. For example, the modeler might calibrate the model to available data (e.g., trends in county health assessments, national surveys), make structural model changes to reflect participants’ mental models, and run simulations during or between sessions.

Action(s)

Scripts outline the primary GMB actions (i.e., activities) [19]. Scripts can be tailored for group size, focal problem, current implementation phase, and partner preferences [20, 64]. Established scripts are generalizable across projects, but can be tailored to project goals [20, 65]. Additional File 1 identifies scripts that align with common implementation strategies.

GMB projects for implementation span some or all of the following (often non-linear) arc [66, 67]: 1) learning about systems thinking — a systematic, “dynamic way of thinking” that is more likely to accurately characterize the root causes of a problem compared to traditional linear thinking [17, 68]; 2) applying systems thinking to complex problem solving; 3) developing and using system dynamics models to identify leverage points that drive system behavior and should be targeted by interventions [69]; and 4) identifying, operationalizing, and selecting implementation strategies by modeling how they could improve implementation and health (i.e., specifying mechanisms) [70]. Steps 3 and 4 could be modified based on the focal implementation phase.

Action targets

GMB action targets are the behavioral and cognitive phenomena hypothesized to change within implementation partners during GMB that subsequently impact implementation quality [71]. GMB encourages individuals to make their mental models — personal interpretations about how the world works — explicit [7, 67, 72]. For example, GMB may improve leaders’ understanding of the focal problem(s) (individual-level), alignment of leaders’ mental models (individual and group-level), and implementation commitment (individual-level and group level). Improved mental model alignment and commitment, in turn, may foster timely intervention adoption, effective implementation planning, and sustained implementation [73,74,75,76,77]. Further, GMB aims to foster systems thinking — a potential mechanism for identifying and adopting appropriate, acceptable interventions.

Dose

The dose or length of GMB projects can be as short as a half-day workshop or extend to multi-hour sessions over longer periods (e.g., weeks, months) [7, 64]. Group size and diversity will inform dosage [7, 64], as will project goals and resources.

Temporality

The Exploration, Preparation, Implementation, and Sustainment (EPIS) implementation framework describes implementation as occurring across four phases, often non-linearly (i.e., activities occur simultaneously across phases, are delayed, or revisited) [32, 51]. GMB is particularly well-suited for supporting exploration and preparation activities — phases for which implementation strategies are not nailed down [51] — including defining problems, adopting interventions, planning for implementation given context (i.e., the system’s dynamics), and selecting, tailoring, or adapting implementation strategies [78]. While other methods are typically suited for selecting and operationalizing either interventions or implementation strategies, GMB can be used to complete both activities. For example, GMB-developed models can inform intervention selection and operationalization by explicating problem dynamics [69] and whether, or how, evidence-based practices (EBPs) target interconnected determinants and feedback loops — closed connections that shape system behavior through reinforcing (escalating or de-escalating) or balancing processes [12] — driving problem dynamics. Models can also explicate how implementation contexts (e.g., barriers and facilitators) may dynamically change over time, thereby informing implementation strategy operationalization, timing, and sequencing. Anticipating system changes is necessary for implementation, as some barriers may be addressed through one strategy, but changes in that barrier may trigger the need for another strategy. For example, workforce turnover is a common implementation challenge [79, 80]. Turnover affects how many individuals can be served and will have varied impacts over time as a new clinician gradually increases their workload, requires less supervision, and increases alignment with the organizational culture and climate [79, 80]. Quantitative simulations, particularly those calibrated with local data, can help decision makers anticipate how and when limited resources should be deployed to mitigate undesirable impacts of such changes. Alternative implementation strategy selection approaches typically rely on lists of barriers, facilitators, and structural supports that minimally account for these dynamics [81, 82]. Regardless of data specificity to context, GMB-developed simulations can identify the patterns of systemic change that may occur during implementation [83, 84].

During implementation and sustainment, models can help plan effective responses (i.e., new or modified implementation strategies) to unplanned events such as withdrawn funding, political push-back, or natural disasters. For example, in implementation, models could simulate whether unplanned changes in Medicaid eligibility would result in a clinical hiring need or whether current capacity could absorb newly eligible patients. In sustainment, modeling could compare impacts on financial solvency of system behaviors such as waitlist growth across alternative training strategies (e.g., onboard multiple clinicians at once versus a staggered approach). GMB-generated products, such as models and materials describing them, can also be used for dissemination.

Implementation outcomes

GMB is hypothesized to primarily affect implementation outcomes such as adoption [29, 33,34,35], appropriateness [35], feasibility [85, 86], reach [37, 38], and sustainment [44, 87]. However, few studies report long-term outcomes [39, 88]. A 2023 systematic review of 72 studies leveraging GMB for implementation [11] reported outcomes such as model confidence (which could affect model appropriateness, reach) and positive system changes. Most outcomes measured thus far may be best characterized as outcomes that are causally (e.g., mechanisms) or temporally preceding implementation outcomes (e.g., adoption, fidelity, acceptability).

Theoretical justification

Three primary theoretical streams underlie GMB: systems thinking, participatory or engaged research, and decision science theories and frameworks such as social choice theory [39, 61, 89, 90]. GMB uses system dynamics models (causal loop diagrams (CLDs) and simulations) and scripts to foster systems thinking. Models illustrate how interconnections between factors lead to emergent behaviors [91] over time that are not easily intuited without simulation. Therefore, GMB hypothesizes that simulation models function as “boundary objects” — visual representations of reality — to develop shared language and mental models, [6, 7, 22, 67]. Shared language, in turn, facilitates trust and agreement about actions through attitude alignment, as detailed in the theory of planned behavior [92]. GMB may overcome barriers to model uptake and action with model insights by teaching participants the underlying methods and theories, and by co-developing the model with participants [35, 93].

Similar to participatory research principles, GMB emphasizes the need to honor GMB participants’ expertise as legitimate and critical for model validation. Engaged perspectives elevate community and practice-based knowledge to an equal or higher status than scientific knowledge. Researchers must critically reflect on power dynamics that shape problems, solution adoption, and implementation processes [44, 64, 94,95,96,97]. Active listening and pausing while facilitating sessions can support such reflections. Community facilitators, as described above, can also help address power imbalances.

Social choice theory is the inquiry (and set of models, theories, and frameworks) of collective decision-making processes and outcomes [59]. Group decisions need support in information processing, communication facilitation, and process structuring [61, 98]. GMB is hypothesized to provide these supports to achieve timely, transparent decisions likely to achieve their intended impact [29, 61]. For example, social choice theorists demonstrated social and mathematical challenges in accurately representing individual preferences in a collective decision, such as elections [59, 60]. GMB uses voting procedures (e.g., “Dots” script) that can reduce response bias and statistical noise often observed when aggregating votes [59, 99]. More importantly, GMB aims to align mental models so that participants are more likely to reach a shared conclusion (i.e., voting for the same option), thereby minimizing the need for synthesizing incongruent votes. Further, GMB can foster learning about how to make implementation decisions aligned with local contexts, which can confer benefits beyond the initial GMB-supported effort.

Methods

Case study

A pilot study aimed to support decisions regarding child maltreatment prevention EBP adoption for local (e.g., community, county) implementation. GMB was identified as an appropriate methodology because system dynamics focuses on defining how behaviors, such as child maltreatment, are the result of self-reinforcing or balancing behaviors (feedback loops) and structural determinants [100]. It is critical to understand how these determinants are interconnected to anticipate intervention consequences (e.g., whether a singular intervention might be insufficient or cause iatrogenic effects). Child maltreatment prevention EBPs typically focus on individual-level determinants, such as parenting skills, yet rarely target structural factors (e.g., social determinants of health such as health care access, stable and purposeful employment) that influence how parents might apply those skills. For example, parents may experience greater challenges in applying evidence-based parenting approaches when they experience mental health distress (due to mental health care barriers) or have minimal time to practice new approaches due to a heavy work schedule (structural employment challenges). EBP appropriateness should thus be defined by both the evidence base and whether the EBP targets prevalent multi-level determinants (e.g., community strengths, needs). EBP acceptability increases when decision makers perceive the EBP to address their priorities in desirable ways. Thus, to address child maltreatment and prepare for successful implementation, it is critical to identify which contextual determinants (e.g., community strengths, needs) are addressed, or not, by each potential EBP. It was hypothesized that GMB would improve actual and perceived alignment (appropriateness and acceptability, respectively) of the adopted EBP with local strengths and needs [101], and provide an efficient method for partners to narrow down which EBPs to consider.

Table 1 describes case study activities. The text below focuses on high-level, practical aspects.

GMB team and participants

The core modeling team — two doctoral-level trained system dynamics methodologists (KHL, LF) and a system dynamics doctoral student (GC) — engaged eight participants from North Carolina (NC). Priority was given to recruiting participants from diverse locations within NC (e.g., multiple counties) who had practice-based experiences as clinicians or administrators at child and family-serving organizations, as these individuals were identified as having professional goals and responsibilities aligned with the pilot project’s goal to identify interventions for child maltreatment prevention aligned with context. Participants were not in a pre-existing decision-making group. Participants or their organization were compensated $350.

Participants (n = 8) included a group home/foster care hybrid model administrator, a children’s advocacy center director, a county non-profit administrator, social workers, a school support specialist, a certified EBP facilitator, and a non-profit director. Five participants were parents. They were mid-career professionals and primarily identified as female (n = 6). One identified as LGBTQ. Participants primarily identified as White Non-Hispanic; one identified as Black.

Planning GMB sessions

The modeling team used ~ 8 h to plan overall session flow and assign roles. Planning involved identifying the number of sessions, session frequency and length, activities, and goals (Additional File 2). Each session required approximately 3 additional planning hours to consider: 1) reasonable participant time expectations, 2) time the modeling team required to distill information and build models between sessions, and 3) desired implementation and research products. The modeling team scoped the project to preventing child maltreatment in NC; participants later defined child maltreatment and prevention (Additional File 4). Figure 1 depicts how research and implementation products were co-developed across sessions.

Fig. 1

Individual and Group Session Output and Facilitation Team Preparation. Legend: Output from each group model building session is indicated in squares. Italicized text between sessions indicates facilitation (research) team actions to prepare for each session

Actions iterated between the GMB principles of divergence and convergence [7, 20]. Divergent activities focused on eliciting heterogeneous insights and expanding understanding of child maltreatment risk. Convergent activities synthesized participants’ insights, priorities, and decisions.

GMB sessions

Participants completed one individual session prior to three group sessions from June 2018 through February 2019. Individual sessions and Group Session 1 occurred in person. Group Sessions 2 and 3 occurred virtually to make participation less costly and more feasible (e.g., reduce travel time).

Individual sessions

First-order goals were to develop a shared problem definition (e.g., child maltreatment) and to make participants’ mental models explicit. Individual interviews, facilitated by the lead modeler (GC), provided participants opportunities to share their mental models without being influenced by peers who might have had different priorities, power, experiences, and expertise [102]. Interviews occurred at a place of the participant’s convenience (e.g., workplace). The modeler first reviewed fundamental systems science concepts, posed the research objectives, and asked partners to share their definitions for four key concepts: systems, child maltreatment, maltreatment prevention, child well-being. The modeler then guided participants to share their mental models using CLDs. A visual presentation (e.g., PowerPoint) introduced additional systems concepts such as aggregation and feedback loops. The modeler pointed out parallels to the participant’s CLD. The participant then created a second CLD focused on a subset of factors they wanted to prioritize from their first CLD. While creating this second, more detailed CLD, participants were prompted to provide stories behind factors’ interconnections and feedback loops. After the session, the modeler refined CLDs with phenomena that were discussed but not drawn. Interviews closed with participants sharing their values around child maltreatment prevention, and their hopes and fears for the project [21].

Group session 1

The first group session had three objectives: 1) introduce participants to one another, 2) establish project definitions of the four concepts explored during individual sessions (systems, child maltreatment, maltreatment prevention, child well-being) and a project vision, and 3) expand, refine, and correct (i.e., validate) a CLD that synthesized individuals’ CLDs. To prepare for objective two and distill a project vision statement (e.g., purpose), the lead modeler blended the individual definitions for the four key concepts and reviewed notes about participants’ values and hopes. To prepare for objective three, the modeling team created a “loop story table” [12] describing partial or complete loop behaviors (Additional File 3).

To increase a sense of shared purpose amongst participants, the first activity reviewed their blended definitions (Additional File 4), vision statement, and prevention framework (Additional File 5). Participants suggested minimal changes and reached consensus on revisions.

Next, the modeling team oriented participants to the synthesized CLD by highlighting four salient feedback loops from the individual sessions: parenting skills knowledge; child welfare system involvement; low pay employment and childcare; and trauma treatment availability. Participants had access to a poster-sized print of the CLD and a web-based version.¹ The web-based version was strongly preferred due to the ability to isolate and zoom in on factors and connections.

The synthesized CLD served several implementation exploration and preparation functions. It 1) introduced participants to others’ mental models so that they could identify how their mental models might align or could be updated — a critical step for activating the mechanism of mental model change, 2) visualized a dynamic hypothesis for how child maltreatment is perpetuated in the local context and might be prevented through EBPs, and 3) highlighted structural factors (e.g., barriers and facilitators) that might impact EBP implementation and be addressed through implementation strategies (e.g., financial incentives, transportation infrastructures). Although the CLD included factors reflecting local contexts, generalizable variable names were used (e.g., public transportation, child welfare placement) to increase CLD transferability to other contexts.

The “Behavior Over Time” script [21] expanded the CLD. This script was tailored for the project by asking participants to draw graphs about an adverse (feared) trajectory and a positive (hoped) trajectory for how a child maltreatment risk factor of their choice (e.g., youth exposure to substances) might change in their context over a discrete time frame (e.g., years). This activity served the dual purpose of identifying factors to be targeted through an EBP and providing informal trend data against which a simulation model could be validated.

Group session 2

This session aimed to gather participant feedback on an initial simulation structure (Fig. 2) and to select EBPs to model. Translating the CLD into a structure of “stocks” and “flows” is a standard step when creating a simulation [12]. Stocks represent aggregated variables (e.g., individuals in treatment). Flows are the processes through which levels change (e.g., hospital discharge rate) [17]. During translation, incomplete feedback loops in the CLD or inaccuracies can become exacerbated and limit the simulation’s validity [12, 64]. EBPs had to be selected during this session to ensure that the simulation structure incorporated both the leverage points targeted by EBPs (e.g., parent–child interaction quality) and leverage points important to participants but not otherwise targeted by EBPs (e.g., family crisis support, transportation). Participants’ perceptions of which EBPs should be adopted were for a hypothetical decision. However, they faced similar decisions through their professional responsibilities.

Fig. 2

Initial Stock and Flow Simulation Structure Presented in Group Session 2. Legend: Stocks are represented in boxes. Flows are represented by double-lined black arrows. Single-line arrows represent connections between ancillary variables and flows or stocks

To prepare for this session, the modeling team translated the synthesized CLD into a stock and flow structure. Key decisions to establish the structure included: which type(s) of maltreatment to focus on; the analysis unit (household, child, or community), and how to define child and family outcomes. The modeling team selected child neglect as the focal maltreatment type for the project’s available resources and timeline because: 1) all factors prioritized by partners could be incorporated; 2) factors associated with neglect are almost always associated with those related to physical and emotional abuse, thereby priming the model to incorporate additional types of maltreatment, and 3) neglect is the most prevalent type of child maltreatment [103,104,105,106,107]. Consistent with the goals agreed upon in Group Session 1, seven EBPs categorized by an EBP registry as primary prevention programs for neglect [108] were identified for prioritization. The modeling team prepared EBP descriptions (Additional File 6) and distributed them electronically one week before Group Session 2.

During the session, the modeling team first explained their rationale to focus on child neglect. Participants found this agreeable. Subsequently, modelers led discussion about the simulation structure and decision points, including which outcomes to measure with stocks. Participants asked clarifying questions, such as whether families with previous child welfare involvement were modeled separately. Key questions about the model’s plausibility and representation of real-world phenomena (i.e., structure assessment, parameter assessment [12] were posed to participants. For example: Were flows realistically impacting stocks? How should risk factor accumulation be measured? Which protective factors should be included? Participants’ questions and clarifications resulted in an updated CLD (Fig. 3), feedback loop stories (Table 2), and simulation structure. Participants’ engagement with the CLD and simulation model’s structure was critical to ensure model structural validity and boundary adequacy [12, 61], and to increase model acceptability to partners. Feedback loops were further validated with the scientific literature and content experts (PL, LS; Table 2). This step is not required, as participants’ expertise is already a type of model validation. Lastly, participants prioritized three EBPs to simulate using a three-step process (Additional File 2) [21].

Fig. 3

Iterated Causal Loop Diagram of Child Maltreatment Determinants Informing System Dynamics Simulation Model. Legend: Arrows of the same color comprise a feedback loop. The colors correspond to the feedback loop label (e.g., R1, B1) with the same colored arrow around it. + signs indicate a positive (reinforcing, either virtuous and desirable or vicious and non-desirable) relationship, in which variables move in the same direction (i.e., An increase in variable 1 causes an increase in variable 2). Key reinforcing loops are indicated with an “R” inside an arrow.—signs indicate a negative (balancing) relationship, in which variables move in opposite directions (i.e., An increase in variable 1 causes a decrease in variable 2). Key balancing loops are indicated with a “B” inside an arrow. Italicized variables are repeated across the diagram for clarity

Table 2 Loop Story Table— Key Feedback Loops and Corresponding Stories of Feedback Behaviors That Affect Child Maltreatment Risk

Group session 3

The primary goal was to obtain additional feedback on the simulation structure—akin to member checking in qualitative research [154]—in preparation for parameterizing the model to simulate the three EBPs’ impacts on target outcomes [155]. Prior to this session, the modeling team updated the CLD and simulation structure based on Group Session 2 feedback. The updated CLD was presented to demonstrate changes and receive corrections. The session primarily focused on the simulation structure, as it had been substantially changed in response to feedback about how best to track neglect-related outcomes. Participants were asked about parameters’ reference modes (i.e., baseline trends) such as parent peer-support and stress, as well as structural decisions such as how to incorporate parents’ trauma history and the impact of positive parent–child interactions — two key targets identified by participants in individual and group sessions. Due to the pilot project’s limited time, additional steps to validate and calibrate (ensure that the model reproduces observed trends) the stock and flow structure for a quantitative simulation were conducted without participants. Thus, these steps are not reported here.

Results

GMB as the action for “conduct local consensus discussions”

Participants agreed that several phenomena were crucial to understanding dynamic child maltreatment risk and thus which interventions might mitigate risks: 1) multi-level trauma, including intergenerational trauma and trauma experienced by providers who interact with families, 2) parent stress due to emotional stressors or basic needs deprivation, 3) availability of mental health treatment for parents and children, 4) parental substance misuse, and 5) parent social supports, especially peer and crisis support. This list of potential intervention targets helped researchers and participants narrow the potential EBP list to a smaller, more manageable set, demonstrating the value of GMB as a planning implementation strategy. One prioritized EBP was not initially considered because it was not considered primary prevention in the EBP registry. It was added due to participants’ strong preference for a program with a peer support component [156, 157].

GMB helped prioritize target factors and developed a boundary object (the CLD) that demonstrated why each factor would be critical for maltreatment prevention. A follow-up study created a brief video (linked in Additional File 2) describing the key feedback loops driving maltreatment risk and how each prioritized EBP did or did not target risk factors.

“Building a coalition” with GMB

The collaborative, consensus-driven GMB process fostered trust and relationship building amongst GMB participants, as indicated by how readily participants asked questions of the modeling team and one another. Participants also discussed potential collaborations during the in-person session.

“Model and simulate change” with GMB

Similar to previous community-engaged modeling projects, participants’ insights about the risk and protective factor interconnections improved dynamic hypothesis accuracy [49, 67]. For example, participants pointed out that parent stress and trauma can have both direct and indirect effects on child behavior, whereas the modeling team had only modeled the indirect pathway. None of the considered EBPs addressed all key feedback loops (Table 2). Instead, participants’ insights highlighted how child maltreatment risk is a complex phenomenon that will require complementary interventions across multiple child and family serving systems to sufficiently address the interdependent causes over time. Delineating the feedback loops that drive child maltreatment was thus critical to prioritize which EBPs were most appropriate and acceptable for the hypothetical implementation context.

Discussion

GMB is a promising implementation approach for operationalizing a single implementation strategy or achieving interdependent objectives of multiple implementation strategies. Just as Hawe and colleagues suggest that health interventions are events that cause changes in complex ecosystems (e.g., healthcare settings or communities) [158], implementation science might benefit from conceptualizing implementation strategies as events within a complex implementation process. These “events” are interconnected such that the whole (context) is greater than the sum of its parts. Implementation strategies dynamically change the implementation context. Therefore, implementation approaches must account for how the system adapts to previously delivered interventions, strategies, and contextual changes [159, 160]. Implementation scientists should partner with implementers to identify how strategies will address interconnected implementation determinants and effects that emerge in the implementation context as a function of implementation processes. Implementation scientists should not merely match implementation strategies to determinants, but also attend to the fundamental objectives of each strategy and how sequencing or co-delivering implementation strategies impacts a system of implementation determinants and emergent behaviors.

Systems science methods, such as GMB, are ideally positioned to support strategy and intervention selection in both implementation theory and practice. In the case study, the biggest leaps in participant learning during GMB come through interacting with systems models and applying systems thinking to understand root causes of observed behaviors — something others have found [35, 87]. For example, while drawing their CLD, one participant realized that they had not been considering the integral role of transportation in another implementation effort. They decided to bring it up during their next workgroup meeting. Whereas other methods and frameworks, notably Implementation Mapping [161] and Implementation Research Logic Model (IRLM, [162]) might match implementation strategies to static determinants, GMB can help identify which determinants dynamically drive implementation success over time, thereby streamlining resource allocation or informing adoption of both innovative and evidence-based strategies. This case study also illustrates how formally modelling dynamic processes can be instrumental for identifying necessary upstream interventions to address a complex problem such as child maltreatment (e.g., parental stress, workforce availability). By consulting the CLD, participants saw how innovative interventions must accompany EBPs to fully address parental stress, and how doing so could affect subsequent risk factors (e.g., parent–child interaction quality).

The case study and empirical literature [29, 39] demonstrate how GMB can effectively impact mechanisms — such as communication quality, trust between practitioners and decision-makers, and motivation to change behaviors — hypothesized to be common across implementation strategies [2, 24, 163, 164]. Strong relationships, social networks, and leadership are essential to implementation success [77, 88, 165]. Investing in methods such as GMB during exploration and preparation can promote trust and communication between implementation partners that will facilitate effective implementation and sustainment. Participants’ connections and relationship quality) can affect group dynamics in sessions, and thus which insights can be elicited. Thus, measuring social network changes and participant involvement throughout GMB could further inform how GMB can be used to “build a local coalition.”

Practical lessons for using GMB

Consistent with other collaborative simulation methods [163, 166,167,168,169] and GMB projects [88], flexible delivery of GMB and building participants’ trust were critical to the case study’s success. Individual sessions were valuable for researcher-participant rapport building. The modeler became attuned to participants’ logistical preferences. For example, some participants preferred sticky notes to create CLDs, whereas some preferred to draw directly on paper.

Diverse scripts enhanced engagement quality and model accuracy. The Behavior Over Time script [21] evoked stories, potential interventions, and factors not discussed in individual interviews or group review of the synthesized CLD. These included: adverse childhood experiences screening, preschool expulsion, children using illicit drugs, subsidized childcare, trauma-informed practices availability, and child welfare involvement timing and intensity.

Alternative meeting modalities, such as virtual sessions, were responsive to participants’ needs yet allowed for research goals to be met. Researcher-participant rapport established during the in-person sessions allowed the subsequent virtual sessions to be engaging. The modeling team employed three procedures to facilitate virtual delivery: explaining logic for the transition, using video conferencing with screen-share, and sharing documents beforehand. While the pilot study occurred before COVID-19, the pandemic encouraged the development of new tools and practices to effectively deliver GMB online [170, 171].

Limitations

Most GMB projects can and should begin with the community identifying the problem(s) they wish to solve. However, it is not uncommon for researchers to approach potential partners and propose GMB [23, 67, 172]. Since this project was researcher-initiated, some process insights may differ from those within established academic-practice partnerships. Yet, the quality of conversations and engagement among partners speaks to the strength of GMB for fostering collaborations and insights. Thus, GMB could also be a method to operationalize the implementation strategy to “develop academic partnerships [173].” Similarly, given that participants were not a decision unit, they did not need to reach decision consensus. Thus, this case study cannot address the time required to foster mental model alignment sufficiently for a group to reach consensus about intervention selection. Future studies should explicitly measure group-level mechanisms such as mental model alignment and consensus to understand GMB’s impact on implementation outcomes such as adoption. Finally, although participants had deep familiarity with parents’ complex needs and strengths, they did not have lived experience as parents involved with child welfare. Engaging parents who participated in parenting EBPs could help identify risk and protective factors not addressed through EBPs.

Future research

Future work could explore which contextual factors affect the appropriateness of GMB compared to other implementation strategy selection approaches. For example, Implementation Mapping offers a structured process for identifying and selecting implementation strategies [82]; however, there is less focus on mental model alignment and no complexity modeling [174].

There is a growing evidence base of GMB evaluations and related measures [11, 87, 88, 175, 176]. Research should evaluate GMB’s impact on other individual and group-level mechanisms such as systems thinking. Additional File 7 includes recommendations for reporting core GMB processes and project characteristics, modified for implementation scientists based upon recommendations by Rouwette et al. (2002) [39]. These recommendations could be systematically expanded with implementation science theories to further specify the potential impact of each GMB planning decision on hypothesized mechanisms and outcomes [177].

Future work can explore GMB appropriateness for operationalizing additional implementation strategies. For example, while “facilitation” enjoys a strong evidence base [178, 179], GMB might be well-suited to operationalize specific components (e.g., establishing a vision, identifying implementation determinants) using systems science. There are numerous parallels between the action targets of GMB and facilitation, including strong partnerships, problem identification, action planning, and priority setting [178]. GMB also shares similarities with the oilcloth implementation planning strategy, during which a facilitator guides learning and conversation using a boundary object [180].

Conclusion

Without rooting implementation planning in systems thinking, implementers are at risk for identifying “fixes that fail.” Moreover, assuming that individuals have a shared understanding of a problem and its solutions (i.e., mental models) will lead to challenges when implementing and sustaining innovations. As one partner noted: “lack of understanding…connects to everything.” By fostering deep understanding of the problem and potential responses using systems thinking, GMB can cultivate commitment to implementing fixes likely to succeed and avoid what one participant described as getting “distracted by the shiny.”