Research – Practice Partnership: Application to Implementation of Multitiered System of Supports in Early Childhood Education

Purpose: Research – practice partnerships (RPPs) offer a tool for identifying, generating, and implementing evidence-based strategies that improve practice, policies, and client outcomes. We offer a description of RPPs and elements that facilitate successful utilization. The origin of RPPs, various approaches, and anticipated barriers are discussed. We illustrate some of the challenges and rewards of establishing an RPP using an example of a project that sought to implement a multitiered system of supports among a variety of early childhood education classrooms in a large urban school district. Method: A planning/leadership team established objectives and used surveys, focus groups, classroom observations, professional development records, and student outcome data to inform decision making. Results: The district ’ s progress in implementing multitiered system of supports in early childhood and ways in which implementation plans were modified as a result of data-based decision making are described. A number of unexpected obstacles interfered with original plans, requiring significant revisions in our theory of change and new strategies to overcome challenges. Conclusion: This illustration helps elucidate critical elements of RPPs and highlights their applicability to researchers and practitioners in communication sciences and disorders.

T he growing recognition of the gap between research and practice in communication sciences and disorders and other disciplines has sparked an interest among researchers and practitioners in implementation science (Bauer, Damschroder, Hagedorn, Smith, & Kilbourne, 2015;Goldstein & Olswang, 2017;Olswang & Prelock, 2015).Implementation science seeks to improve the uptake of empirically supported practices into everyday service delivery in real-world settings (Douglas & Burshnic, 2019;Eccles & Mittman, 2006).A hallmark of implementation science is collaboration among researchers and practitioners (Olswang & Goldstein, 2017).Researchers are realizing that developing innovative clinical and educational practices and establishing their efficacy are necessary but not sufficient.
Intervention research needs to be coupled with implementation development to reduce the many years typically required for evidence-based practices to be applied routinely in everyday practice (Goldstein & Olszewski, 2015;Green, Ottoson, Garcia, & Hiatt, 2009).To this end, communication science and disorders researchers are seeing the value of working with practitioners.Likewise, practitioners are appreciating the value of conducting research in their settings to improve clients' outcomes and to demonstrate accountability for their services provided to individuals with communication disorders (Campbell, Camden, & Missiuna, 2016;Crooke & Olswang, 2015).
This article describes and illustrates a productive means of collaboration through research-practice partnerships (RPPs).RPPs have attracted the attention of a number of funding agencies, such as the Institute of Education Sciences (IES, 2018), the Spencer Foundation, and the W. T. Grant Foundation, (2018).The ASHFoundation has a new initiative to award researcher-practitioner collaboration grants to support partnerships seeking to investigate approaches to enhance assessment and treatment practices that hold promise for improving services for individuals with communication disorders.The American Speech-Language-Hearing Association also has sought to encourage the formation of collaborations between clinicians and researchers.To assist with that matching process, the American Speech-Language-Hearing Association established an online tool, CLARC, which stands for Clinicians and Researchers Collaborating.Consistent with such initiatives, the objective of this article is to provide speech-language pathology and audiology researchers and practitioners with the opportunity to learn more about collaborative partnerships.We provide an example of an RPP to illustrate how this model may be incorporated in settings where collaborative research holds potential for contributing unique knowledge to the field and improving outcomes for the stakeholders we interact with and the clients we serve.

Definition
RPPs are collaborations among researchers, practitioners, and members of the community to examine a topic of interest, enhance the quality of research, and generate approaches or solutions to address problems of practice (Coburn, Penuel, & Geil, 2013;Minkler, 2005;Ralston, Tarasawa, Waggoner, Smith, & Naegele, 2016;Ralston, Weitzel, Waggoner, Naegele, & Smith, 2016;Reardon & Leonard, 2017).They also may be referred to as researcherpractitioner partnerships (Farrell et al., 2018) or practitionerscientist partnerships (Spoth & Greenberg, 2005), and they share much in common with community-based participatory research or action research (Alegria et al., 2011;Lewin, 1946).Drahota et al. (2016) used the term communityacademic partnership in their systematic review of the literature.RPP is a term that seems to have been embraced in education but also has been used across a variety of disciplines, such as public health, medical fields, criminal justice, and business (Christopher, Watts, McCormick, & Young, 2008;Minkler, 2005;Spoth & Greenberg, 2005;Stokols, 2006;Westfall, Mold, & Fagnan, 2007).All of these approaches fit within the broader realm of implementation science.But implementation research has a different emphasis, as it focuses on studying the strategies used to adopt and integrate evidence-based practices into clinical, education, and community settings to improve outcomes of various populations.RPPs are more than simply translating research to practice.RPPs allow for productive relationships to be built among researchers and practitioners whereby their active involvement generates joint research objectives and plans for change within the organization, benefiting all those involved and allowing for an iterative colearning process (Israel, Eng, Schulz, & Parker, 2005;Spoth & Greenberg, 2005;Tseng, Easton, & Supplee, 2017).Flexibility and adaptability are necessary for connecting science and practice so that each area still maintains key elements as they gradually grow together (Riemer, Kelley, Casey, & Haynes, 2012).The conceptual definition put forth by Drahota et al. (p. 192) states: "Community-academic partnerships are characterized by equitable control, a cause(s) that is primarily relevant to the community of interest, and specific aims to achieve a goal(s), and involves community members (representatives or agencies) that have knowledge of the cause, as well as academic researchers."

Origin and the Evolution of RPPs
RPPs seemed to be foreshadowed by Lewin (1946), who suggested that action research conducted with participants was needed.He proposed a research cycle: planning, taking action, and investigating the results of the action (Wallerstein et al., 2003).Parsons (1950) encouraged decision making through application of scientific knowledge to practical problems.These perspectives are consistent with the realization that researchers must work with practitioners to ensure that their research is relevant, has utility, and is feasible for implementation in real-world settings (Drahota et al., 2016).A number of disciplines have established RPPs to improve the development of intervention practices and to facilitate implementation of those practices that have strong empirical bases.
As accountability for evidence-based practice and data-based decision making increases, RPPs have the potential to yield more impactful research questions and better methods for producing evidence.This not only improves services but also ultimately improves practices that are employed and sustained in real-world settings and relevant to needs within the community.RPPs have sought to facilitate implementation of evidence-based decision making among educators, for example, as partners typically seek to generate solutions leading to improved district-wide outcomes (Coburn et al., 2013;Wentworth, Mazzeo, & Connolly, 2017).Although RPPs offer an auspicious approach to enhancing the function of research to advance educational practice, empirical reports of effects of this approach are limited (Coburn & Penuel, 2016).
The National Institutes of Health, the Centers for Disease Control and Prevention, the Institute of Medicine, and the Patient-Centered Outcomes Research Institute have recognized the importance of researcher and practitioner collaboration to improve the implementation of research into practice and to decrease inconsistencies in access to and quality of care (Drahota et al., 2016;Garland, Plemmons, & Koontz, 2006;Patient-Centered Outcomes Research Institute, 2012).The Council for the Accreditation of Educator Preparation added a new standard related to partnerships in 2013, noting that educator preparation providers need to develop successful and beneficial partnerships with the community in P-12 schools (Ralston, Weitzel, et al., 2016).
There is much to be learned from RPPs and how they have evolved.There has been a historical shift from transactional partnerships with an absence of mutual purpose to transformational partnerships that mutually benefit researchers and practitioners (Ralston, Tarasawa, et al., 2016;Ralston, Weitzel, et al., 2016).For example, Hasche et al. (2014) point out that partnerships among stakeholders may involve researchers who are interested in bringing evidence-based interventions to new settings.However, researchers must realize that practitioners are key players to help us understand how treatment must be adapted to different circumstances.Treatment adaptation and how it relates to treatment fidelity is an underdeveloped topic in implementation science.
RPPs are often formed based on a previous partnership to respond to additional needs that have been identified (Johnson, Severance, Penuel, & Leary, 2016).They may be based on results of a needs assessment to determine an area of importance within a community (Wilson, Strayer, Davis, & Harden, 2018a).They may involve a variety of individuals, for example, teachers and school leaders (Johnson et al., 2016;Wilcox, Lawson, & Angelis, 2017), health educators and behavioral and implementation scientists (Wilson, Strayer, Davis, & Harden, 2018b), and even partners with varying roles and differing knowledge from multiple countries (Jesson & Spratt, 2017).These individuals often are part of a leadership team that may have several responsibilities (Hasche et al., 2014;Jesson & Spratt, 2017;Wilson et al., 2018b), such as

•
setting goals, • discussing project progress, • engaging in problem solving related to practice, • providing feedback on data collected to analyze practice, and • working together to identify appropriate evidenceand practice-based adaptations.
Thus, the individuals who form RPPs work together to identify evidence-based interventions that meet the needs and the presumed realities of the practice settings.The partners within the community or agency must strive to develop an adaptable model that leads to acceptability, feasibility, and sustainability (Hasche et al., 2014;Harden, Johnson, Almeida, & Estabrooks, 2017;West, Aparicio, Berlin, & Jones Harden, 2017;Wilson et al., 2018a).The scale of these projects may vary widely.Most RPPs involve a single agency.Others may involve larger systems, for example, multiple school districts (Ralston, Weitzel, et al., 2016) or even schools spread across three Pacific Island countries (Jesson & Spratt, 2017).

Types of RPPs
RPPs share some common characteristics, but at this point, one may see wide variations in this approach.For example, Coburn et al. (2013) distinguished among three types of RPPs in education based on the scope of their purpose, objectives, and activities."Design Research Partnerships" are collaborative efforts to design studies, often an iterative series, to investigate innovations in teaching and learning.For example, the RPP might design a study to investigate whether teachers who learn new literacy instructional techniques produce improved student outcomes."Networked Improvement Communities" seek to establish collaborations among researchers, practitioners, and designers organized around a shared problem of practice (Bryk, Gomez, Grunow, & LeMahieu, 2015).The network commits to using shared tools and methods of improvement science to test new practices, gather data, and share what is learned across the network.The goal is to systematically investigate how promising practices are adapted to be effective in a variety of contexts."Research Alliances" typically have a school district or a regional focus typically helping organizations to use data and research to improve academic outcomes for students.For example, the IES funded 10 regional educational laboratories across the country to work in partnership with educators and policymakers to conduct applied research and trainings with a mission of supporting a more evidence-based education system.Hybrid approaches may combine these types of RPPs.As we gain more experience in developing and evaluating RPPs, we might expect an evolution in their approaches.For example, what starts out as a design research partnership may evolve into a networked improvement community.

•
absence of a shared mission, vision, and goals; • sites choosing to no longer participate in a study due to lack of adequate resources, buy-in, or leadership support; • research on evidence-based practices ignoring the realities of the practice setting; • clinicians making requests that cannot be honored because of the research design (e.g., selection of participants); • differences in the way of work for practitioners and researchers; • difficulties in acknowledging new perspectives or coming to a common understanding; • inconsistent attendance of team members at meetings or turnover in team members impeding progress; • miscommunication stemming from differences in the language or jargon used by team members; • poor relationships among team members that impact the work; • perception that additional activities associated with the partnership are burdensome; and • time constraints that are perceived as barriers to full or adequate involvement in carrying out an evaluation and discussing research findings are the most common complaints.Johnson et al. (2016) and Jesson and Spratt (2017) acknowledged the tensions that can arise when engaging in RPP work even when a common vision is shared.Thus, the conceptual framework for partnerships must incorporate a balance between practitioner knowledge and experience and evidence-based knowledge and experience resulting from research.This balance must be sustained for all team members to find value in new ways to look at research and practice and to recognize the knowledge that can be shared among members of the RPP.There may be some partners within RPPs who perceive others as having questionable intentions at times, but overall impressions typically reflect positive experiences.West et al. (2017) emphasize the significance of the personal relationships among partners as well as the rapport among practitioners and families.They also note the importance of explaining how confidentiality of information obtained during the study needs to be protected.Riemer et al. (2012) emphasize the importance of clarifying shared responsibilities and explicitly delineating team roles to resolve conflicts that may arise in a partnership.West et al. (2017) point out a common point of contention when research designs relegate some participants to control groups through random assignment, especially if this can recur in a series of studies.These concerns need to be acknowledged and may encourage discussion of waitlist control groups or alternative treatment comparisons that are likely to benefit participants.For example, Riemer et al. (2012) noted a conflict regarding research design and the feedback given to the control group sites.The team resolution was a design where sites were randomly assigned to receive weekly feedback with 90-day reports versus sites that received only 90-day reports.Ralston, Weitzel, et al. (2016) noted that, even with best intentions, it is difficult to produce a curriculum, professional development, and research projects that yield desired outcomes.Thus, a long-term commitment to iterative development and continuous quality improvement typically should be the expectation.Jesson and Spratt (2017) acknowledged that strains may arise in trying to maintain treatment fidelity, planning replications and scaling up, and evaluating the outcomes of practices on student learning.Johnson et al. (2016) discussed apprehensions related to the selection of tasks to focus on when both common and competing goals present themselves.Addressing tensions promptly may facilitate improved understanding among partners who are striving to reach agreements that balance values, goals, and resources (Johnson et al., 2016).
Many RPPs have successfully incorporated program evaluation, data alignment, and professional development practices to support changes (Ralston, Weitzel, et al., 2016), but sustaining these RPP activities can be a challenge.Ralston, Weitzel, et al. (2016) suggested a number of factors that may help sustain partnerships, including secure funding for a long period, work with leaders both in groups and individually, establish strong communication networks, involve graduate students, strive for proper balance among stakeholders' agendas, and work together to determine the areas of highest need.
RPPs are not always able to overcome barriers to executing rigorous research methods.Consequently, case studies and qualitative methods are used frequently.Future reviews of RPPs will need to provide more complete information on the number of participants and the length of partnerships.Evaluations of the effectiveness of partnerships are needed that include evaluations of the collaborative process, assessments of the goals completed, and analyses of outcomes within the community related to the implementation of intervention, increased knowledge, and sustainability (Drahota et al., 2016).

Framework and Key Elements of RPP
The challenges that we can anticipate do not negate the value and the importance of RPPs as a means of improving service delivery.To that end, authors have described what they consider to be key elements of RPP (Baker, Homan, Schonhoff, & Kreuter, 1999;Henrick, Cobb, Penuel, Jackson, & Clark, 2017).There is overlap in the dimensions addressed in partnerships from the perspective of various disciplines and various stakeholders.For example, Baker et al. (1999) focused on how the partners operate, such as acknowledging and honoring different partner's "agendas," acknowledging the difference between community partner input and active community involvement, developing relationships based on mutual trust and respect, and being aware of partnership maturation and associated transition periods.There have been attempts to identify key elements from the RPP literature to guide attempts to build a foundation for strong partnerships.Henrick et al. (2017) identified and sought to validate five essential dimensions of RPPs: 1.
Building trust and cultivating partnership relationships.

2.
Conducting rigorous research to inform action.

3.
Supporting the partner practice organization in achieving its goals.

4.
Building the capacity of participating researchers, practitioners, practice organizations, and research organizations to engage in partnership work.

5.
Producing knowledge that can inform clinical and educational improvement efforts more broadly.
The first two dimensions seem most critical and serve as prerequisites to the execution of the remaining dimensions.Henrick et al. ( 2017) offer a number of indicators for each dimension to help partners evaluate the extent to which their RPPs exemplify these key dimensions (see Table 1).
RPPs need to be structured in ways that facilitate the realization of the five key dimensions.Partnerships typically emerge from existing relationships.Some may be initiated by a researcher who is disposed to practice-informed collaborations.Others may be initiated by a visionary agency leader who is seeking scholarly expertise to help improve educational or clinical outcomes.Alternatively, opportunities may present themselves through a Request for Proposals for funding that bring colleagues and acquaintances together into an RPP.Regardless, a shared purpose helps shape collaborations between research and practice partners (Olswang & Goldstein, 2017).
Most RPPs benefit from documents that establish joint expectations (Coburn, Bae, & Turner, 2008).These expectations do not necessarily represent a specific research agenda.It is more important to agree on the overriding research purpose and how the partners will work together.The shared expectations could be documented in a memorandum of understanding or agreement, a partnership charter, a set of operating principles, or grant proposals.Such documents would be expected to outline stakeholder representation, roles, governance, and operational issues.A separate data sharing agreement document is typically developed to establish how data are shared and with whom, how data are secured and confidentiality is maintained, and how findings are communicated and with whom.Dissemination plans typically spell out how research findings will be shared among stakeholders, the broader agency community, and the public and scientific community.It behooves the partners to contemplate how both positive results and disappointing results will be handled.

Cultivating Partnership Relationships
The impetus for our illustration of an RPP was a grant opportunity through the IES.A casual conversation between a university professor and a research director in a large school district who had worked together previously on early intervention projects set the stage for a meeting that included a number of district administrators from three segments of the district: supervisors of early childhood Table 1.Five dimensions for assessing research-practice partnerships (RPPs) and associated indicators.

Building trust and cultivating partnership relationships
• Researchers and practitioners routinely work together, participate in well-organized meetings, and communicate effectively.
• The practice organization identifies a problem of practice and researchers work with the practitioners to plan for addressing this problem.
• The RPP establishes routines that promote collaborative decision making and guard against power imbalances.
• RPP members establish norms of interaction that support collaborative decision making and equitable participation in all phases of the work as new information is introduced and integrated or as materials, tools, and resources are developed or revised.• RPP members recognize and respect one another's perspectives and diverse forms of expertise.
• Partnership goals consider team members' work demands and roles in their respective organizations.
• RPP members work toward utilizing what they have learned and carry out their specific roles.
• Roles may change to adapt to the barriers that are present.
• RPPs add to the resources and strengths of a community.

Conducting rigorous research to inform action
• RPP conducts research that addresses problems of practice facing the practice organization.
• The RPP establishes systematic processes for collecting, organizing, analyzing, and synthesizing data.
• Decisions about research methods and designs balance rigor and feasibility.
• The RPP conducts research to clarify and further specify problems of practice prior to identifying and assessing strategies for addressing those problems.• Findings are shared in ways that take account of the needs of the practice organization and the team works together to determine action items and next steps.3. Supporting the partner practice organization in achieving its goals • The RPP provides research and evidence to support improvements in the partner organization.
• The RPP helps the practice organization identify productive strategies for addressing problems of practice.
• The RPP informs the practice organization's implementation and ongoing adjustments of improvement strategies.4. Building the capacity of participating researchers, practitioners, practice organizations, and research organizations to engage in partnership work • Team members develop professional identities that value engaging in sustained collaborative inquiry with one another to address persistent problems of practice.• Team members assume new roles and develop the capacity to conduct partnership activities.
• Participating research and educational organizations provide capacity-building opportunities to team members.
• The work of the RPP contributes to a change in the practice organization's norms, culture, and routines around the use of research and evidence.• There are shifts in professional expectations for education researchers and for practitioners that reward members from each organization for sustained participation in significant partnership work.• The RPP establishes conditions in the practice organization that lead to sustained impact beyond the life of the partnership.
• Research and practice organizations allocate resources to support partnership work. 5. Producing knowledge that can inform educational improvement efforts more broadly • The RPP develops and shares knowledge and theory that contribute to the research base.
• The RPP develops and shares new tools and/or routines that can be adapted to support improvement work in other settings.
• The RPP develops two dissemination plans, one that supports partnership goals and allows for scale up the work in the targeted setting and a second plan for broader dissemination to the research community.

Identifying a Problem of Practice
Because the practice organization determines the area(s) of focus for an RPP, the initial meeting with the district was timely; the supervisors were beginning to plan for the upcoming school year and were eager to learn more about how early childhood education could become involved in MTSS.This is why the school district's research director invited the RtI/MTSS supervisor to the initial meeting.Remarkably, by the end of this first meeting, a team was identified to plan an initiative to improve the fledgling implementation of MTSS in early childhood education programs.
Similar to national trends (Carta et al., 2014), the district had adopted an MTSS framework for problem solving and differentiating instruction for struggling learners in kindergarten through 12th grade, but this approach was not commonly used for Pre-K students.MTSS is a framework that uses data-based problem solving to differentiate levels of instructional support provided to students based on their demonstrated needs (Gersten et al., 2009).It includes screening and progress monitoring as well as tiers of highquality instruction with the intent of preventing later academic and behavioral difficulties.Addressing this implementation need was consistent with the increasing adoption of MTSS in early childhood nationally (Division for Early Childhood, National Association for the Education of Young Children, & National Head Start Association, 2013; Greenwood et al., 2011).This initiative was precipitated by calls by state and district policymakers for greater accountability for student learning results (ECO Center, 2011;Office of Head Start, 2012).
Many of the early childhood teachers in the district had received some training in MTSS.It was incorporated into the beginning of the year orientations in Head Start, for example.Reportedly, an estimated 25% of the early childhood special education teachers had participated in 3 days of training and 16 coaching sessions in an experimental evaluation of MTSS training through the teaching pyramid framework (Fox, Dunlap, Hemmeter, Joseph, & Strain, 2003).Thus, the RPP members sought to work together to identify factors associated with successful implementation and factors associated with less successful implementation.

Meeting, Communicating, and Identifying Roles
Two university researchers, who had worked together previously on the development of a model classroom integrating MTSS across academic and socioemotional domains in a different district, clearly established their objective as helping the district employ data-based decision making.The early childhood education and MTSS supervisors were responsible for deciding how data might inform changes in practices and policies.District administrators responsible for Head Start, early childhood special education, voluntary Pre-K programs (public and private Pre-K), the K-12 MTSS program, and the research department participated in planning.They met weekly for 2 months as part of an ongoing planning team to prepare a grant proposal.
Although the lead researcher facilitated the meetings, it was made clear that the district personnel were responsible for identifying the scope of the project, the data sources that might be most useful for decision making, and the goals for integrating MTSS into the school system.An outline of the required components of the grant application served as the agenda for the meetings.The various sections were topics of the conversation each week as all stakeholders shared their expertise and contributed to the proposal.An updated draft of the grant was sent to the team prior to the following week's meeting to be reviewed and discussed to ensure it accurately captured all elements from the previous week's discussion.The research department representatives served as a critical resource for gathering district information about students, teacher professional development, curricula, assessments, and MTSS procedures that needed to be summarized for the grant proposal.Through this collaboration, a partnership between the university and the school district was established to instill a data-based approach to inform the implementation of MTSS among a variety of early childhood education classrooms in public elementary schools.A memorandum of agreement, which included a data sharing agreement, was reviewed by general counsels at the university and the school district and signed off by the superintendent, dean, and the co-principal investigators (Co-PIs).Despite agreement on the need for implementation of MTSS in early childhood, an examination of current MTSS implementation and potential innovations was undertaken to improve upon the district's MTSS framework (Tout, Metz, & Bartley, 2013).The planning team outlined three teams needed to carry out the main facets of the RPP: (a) leadership team, (b) research team, and (c) school-based problem-solving leadership teams.

Developing Partnership Goals and Overcoming Unforeseen Barriers
The first goal was to establish and convene leadership and research teams.Committee roles of each of the team members were discussed and included in the grant proposal.The proposal also included a partnership process evaluation to monitor perceptions of the success of partnership processes and a fidelity measure to monitor accomplishments of proposed objectives and activities.Thus, we established a process for regularly discussing the purview of committees, how frequently they met, how we would share information, and whether the partnership was meeting its objectives.The Co-PIs from the university and the district had equal responsibility for ensuring that interaction and communication expectations were met.The project coordinator worked under the direct supervision of the assistant superintendent Co-PI but met with the university Co-PI, at least weekly.One unforeseen challenge was the need to replace the project coordinator.This happened twice, as two project coordinators relocated out of the district during the 2 years of the project.Meetings of the leadership and research teams each occurred monthly to provide project updates, discuss data, and determine action steps.Effective lines of communication to facilitate collaboration among the teams and other stakeholders were established to sustain and maintain the teams as new problems or issues emerged.
Despite careful planning to establish this RPP and a good execution of those plans, a number of unanticipated complications interfered with our plans.These events caused the district personnel and university partners to engage in active problem solving as they sought to use an implementation science framework to drive system-wide change.First, our original plans went awry when budget cuts in the district resulted in the elimination of the MTSS facilitator positions and the instituting of a more decentralized administrative structure districtwide.This undermined our plans for having MTSS facilitators introduce Pre-K teachers into the existing elementary school problem-solving teams.
The leadership team set about revising their theory of change to incorporate new plans for facilitating this critical objective.A theory of change offers a graphic analysis of the factors or intervention components that are thought to result in short-and long-term outcomes and illustrates what changes are planned and how they are expected to improve outcomes.The program officer from the IES described the theory of change as a "living document," which helped to buoy the spirit of the RPP.In the second year of the project, working through school principals directly and by implementing job-embedded coaching helped us salvage this goal, but without the widespread implementation originally proposed.In addition, some teachers started to participate in professional learning communities (Hord, 1997;Vescio, Ross, & Adams, 2008) at school sites to engage in active discussion regarding student data, which helped prepare them for their school's problem-solving leadership team meetings.To fully achieve this goal, the RPP members recognized that systemic change is needed (a) to build expectations on the part of principals and school-based problem-solving teams and (b) to provide support (training, materials, technical assistance) for early childhood teachers, possibly pairing them with kindergarten teachers.
The second goal, to implement professional development activities, also proved problematic.Workshops were delivered at a central location in the first 18 months of the project.However, the percentage of teachers attending workshops fell precipitously in the second year.This was exacerbated by a moratorium on expected salary increases, which resulted in teachers "working to the contract."Thus, new strategies were initiated to increase engagement in professional learning.Workshops were replaced by online training modules, communities of practice meetings, and job-embedded coaching.Job-embedded coaching occurred at approximately 20 school sites with teachers who indicated an interest.The project coordinators provided coaching and assisted teachers with (a) reviewing data to identify students needing more intensive instruction and (b) developing intervention plans and accessing materials to implement tiered interventions.District resource teachers participated in some coaching sessions and problem-solving leadership team meetings, which provided an orientation to the kind of practice-based coaching that was needed to support early childhood teachers (Snyder, Hemmeter, & Fox, 2015).Resource teachers met on a monthly basis with the project coordinator to address additional questions, comments, and concerns with this process to prepare them to continue this work the following school year.The communities of practice served a more important function as teachers from the various settings had the opportunity to come together and discuss resources and strategies they use in their classrooms.Teachers followed up by completing a new instructional activity with their students and provided their impressions.
The third goal was to use various district data sources to assist in decision making.University researchers' statistical analyses of those data and reports provoked much discussion about data-based decision making.The district stakeholders were initially discouraged by the results, but this sparked conversation around what types of changes needed to be made to improve outcomes.The assistant superintendent Co-PI began to more closely inspect practices in early childhood and continued conversations around data during biweekly meetings she facilitated with all early childhood supervisors.

Measures
Data sources included focus group interviews with a variety of stakeholders (i.e., related service providers, Head Start teachers, early childhood special education teachers, administrators, paraprofessionals, and parents); perceptions of RtI skills surveys of early childhood teachers; schoolbased MTSS self-assessments; classroom observations using the Classroom Assessment Scoring System (Pianta, La Paro, & Hamre, 2008) and Teaching Pyramid Observation Tool (TPOT; Fox, Hemmeter, & Snyder, 2014); Pre-K student assessments using the Florida Voluntary Pre-Kindergarten Assessment (Florida Department of Education, 2013) and Teaching Strategies Gold (Heroman, Burts, Berke, & Bickart, 2010); and kindergarten readiness assessments using the Hillsborough County Public Schools Kindergarten Reading Test, Star Early Literacy (2017), and i-Ready (2017).

Clarifying and Further Specifying Problems
Before potential solutions could be proposed, we sought to analyze data from many sources.The discussions around teacher preparation and student performance on their respective assessments allowed for further specification of the problems.The data summarized below indicated that teachers needed help in implementing MTSS; student data reinforced the assumption that implementation was rarely occurring with adequate fidelity.

Teacher Preparation in MTSS Practices
The state of preparation of teachers and classroom implementation data came from two sources.The Perceptions of RTI Skills Survey-Revised (Castillo et al., 2013) indicated a need to better prepare teachers to use data.Specifically, teachers needed help calculating the gap between students' performance and benchmarks, using data to adjust core instruction or provide supplemental instruction, determining the appropriate supplemental intervention for students at risk, accessing resources to develop evidencebased interventions, modifying intervention plans, and graphing student data.
The TPOT was the one new measure the district partners chose to try.They sought to determine its utility for assessing classroom practices that support children's social emotional skills and prevent challenging behavior.The research partners gathered baseline observational data in a total of 60 classrooms.Initial TPOT summary reports showed a range of teacher support of social-emotional development.The percentage for the Key Practices subscale ranged from 41% to 96%, and 55% of classrooms met or exceeded the 70% benchmark.Strengths were identified in several areas, including transitions between activities, promoting engagement, providing directions, and collaborative teaming.Further support generally was needed in the areas of teaching social skills, behavior expectations, friendship skills, and problem solving and in supporting family use of practices.Teachers, often with assistance of resource teachers or other staff, were asked to plan activities or training to address areas of need identified in the TPOT reports.Unfortunately, results of posttesting indicated little movement in TPOT scores.It is likely that many teachers need practice-based coaching (Snyder et al., 2015), as the resources provided to assist them with explicit teaching of social-emotional skills proved largely insufficient to change behavior in this domain.

Student Developmental Outcomes
The primary indicators of children's developmental outcomes came from state-or district-mandated Pre-K and Kindergarten Readiness assessments.Head Start and VPK student progress in the areas of language, phonological awareness, print knowledge, and math was assessed using three waves of the state-mandated VPK assessment.The percentage of students exiting district preschool classrooms meeting or exceeding expectations was rather high (82%-90%).However, at the beginning of kindergarten, only 52%, 46%, and 39% of those children met the criteria for kindergarten readiness on the Star Early Literacy, Kindergarten Reading Test, and i-Ready measures, respectively.These findings indicated that the percentage of students entering kindergarten meeting expectations was lower than what might be expected based on the percentage of students meeting expectations on the VPK assessment administered at the end of Pre-K.However, students who exit preschool exceeding expectations on the VPK assessment for phonological awareness were more likely to meet kindergarten readiness expectations, 66%, 65%, and 47%, on the Star Early Literacy, KRT, and i-Ready measures, respectively.These results led to plans for identifying appropriate and predictive assessments for the district's youngest learners.In particular, the types of assessment that most easily facilitate instructional decisions are needed, as there is a misunderstanding of what constitutes progress monitoring tools that can be used to determine whether children are making progress.Teachers need assessments that provide meaningful data to guide instruction and effectively monitor the learning of students who are performing below expectations.

Building Capacity to Sustain Engagement in Partnership Work
As the relationships among members of the RPP strengthened, a mutual trust was established and a spirit of collaborative problem solving among the district and university team members emerged, indicating a good prognosis for successful systemic change.Although student and teacher outcomes have not demonstrated clear improvements during the first 2 years of this RPP, it has sparked constructive conversations among the team members related to improving educational outcomes.These conversations were sometimes tense because the results were disappointing.Nevertheless, developing action steps at the end of each meeting has proven useful as a concrete strategy for moving the work of the RPP forward.This could not have been possible without the development of mutual respect and the recognition that all team members were truly committed to improving student outcomes.
The RPP identified a number of next steps.The partners have discussed establishing higher expectations for teachers and students, selecting common curricula and assessments across all early childhood education programs, preparing resource teachers to use practice-based coaching for supporting preschool teachers, developing more online modules using a "why/how" format, including MTSS in strategic plans, and offering training for elementary school administrators.These initiatives should increase teacher, supervisor, and administrator awareness of expectations for their students during and after they leave the preschool environment and should provide information to assist with improving kindergarten readiness.

Producing Knowledge That Informs Improvement Efforts More Broadly
The research conducted by the RPP has begun to have far-reaching effects on the district and its approach to solving practical problems.This article represents a self-reflective, collaborative exercise in examining the contributions of the RPP.The district is more willing to partner with university collaborators, especially those with a longterm commitment to data-based decision making.This has provided more opportunities for university faculty to share and evaluate innovative curricula and assessments that have the potential to help the district achieve its goals.The broadening and long-term effects of the RPP are evident in several ways.The following strategies are among those incorporated into a revised theory of change for the district, as plans beyond the funding period of the grant are being formulated in at least three areas.
First, the early childhood supervisors, resource teachers, and classroom teachers will be provided with guidance around practice-based coaching to enhance Tier 1 instruction and improve teaching effectiveness.The focus will be placed on what to teach, when to teach, how to teach, and how to evaluate fidelity of implementation and child progress.Suggestions for updating the documentation system for monitoring the visits with teachers will be provided to allow for collaborative partnerships that allow for shared goals and action planning, focused observation, and reflection and feedback (Snyder et al., 2015).The project coordinator will continue to work with the supervisors, teachers, and school sites on implementation of MTSS in early childhood classrooms and involvement of Pre-K teachers in school-based activities related to data discussion and problem solving.The RPP has revitalized efforts to improve MTSS implementation in Preschool-12th grade.
Key decision makers are charged with developing a common language and understanding of MTSS, identifying who has the responsibility for what and how those individuals will be held accountable, ensuring that district policies support implementation of the model, and providing sufficient professional development and technical assistance.
Second, online modules will continue to be made available for the early childhood teachers within the district.Brief modules (approximately 15 min) are being developed to focus on areas of need identified by the teachers, such as descriptions of best practices, how to use data, how to implement tiered instruction, and how to find resources used to implement MTSS.Each module will present reasons "why" teachers engage in particular MTSS practices and "how" they can apply the information from the training into their classrooms.
Third, the RPP members realized they need to engage higher level administrators.Trainings for site-based administrators with early childhood classrooms on their school campus are needed.The administrators will be provided with information to assist with gaining a better understanding of early childhood education and how to integrate preschool into site-based initiatives.Principals and assistant principals will be made aware of what to look for when observing an early childhood classroom.They will be provided with guidelines for incorporating preschool in all events, including faculty meetings, communities of practice, quarterly academic reviews, and problem-solving leadership team meetings at the school site.
It is worth noting that these conversations have influenced priorities at the highest levels of the district.For example, the superintendent has launched a Bold Beginnings initiative to make school readiness more accessible to preschoolers by adding preschool seats across the district.Whereas early childhood education often was separated from the overall culture and administration of elementary schools, a new sense of ownership by principals is the new expectation.This initiative has called for the realignment of all early childhood programs to ensure the vision, goals, values, and principles are clear to all stakeholders.The roles and responsibilities of all program supervisors, school site staff (e.g., administrators, student services personnel), and teachers and teacher aides are being reexamined to delineate how to ensure early childhood students are provided effective learning opportunities.
In general, the innovative nature of this RPP has allowed us to address many issues likely to arise when striving to implement MTSS in early childhood settings.It clarifies the need for trained personnel within the various early childhood programs and at the school sites to monitor and support changes in implementation of effective MTSS practices that are sufficient to bring about meaningful improvements in kindergarten readiness.In addition to the data collected on child outcomes, metrics to assess schools' levels of involvement with their early childhood programs (e.g., number of data discussions and problemsolving meetings) also are needed.

Conclusions
RPPs provide opportunities for stimulating productive interprofessional collaborations by speech-language pathologists, audiologists, and researchers in the field.These professionals may choose to partner with researchers, practitioners, and community members from other fields, settings, or circumstances to study and potentially solve problems of clinical or educational importance.The goals of RPPs may vary.Some choose to examine the implementation of evidence-based practices and frameworks in real-world settings.Others design studies to evaluate whether proposed innovations have desired outcomes that resolve problems.The scale of RPPs may vary.RPPs may be established around the needs of small organizations, large organizations, multifaceted systems of providers, or networks that form an alliance around common problems.Likewise, a vast array of methods is available to serve the purposes of RPPs.It may not be unusual for RPPs to have a lengthy history before their work culminates in sustained implementation of practices and policies that meet the needs of organizations.Those practices and policies then can be adapted to variations in real-world settings and evaluated to ensure that they still show evidence of desired outcomes.The hope is that collaboration among dedicated researchers and practitioners will spur efforts to investigate intervention development and implementation development simultaneously (Goldstein & Olszewski, 2015).If so, perhaps the time involved to develop, disseminate, and implement clinical innovations will be far less than the typical 17 years associated with the traditional research to practice pipeline (Green et al., 2009).
As illustrated, RPPs are likely to be a dramatic departure from researcher-planned experiments, even those undertaken in real-world settings.RPPs set out to bring about implementation of evidence-based practices within systems that are typically complex.Not surprisingly, because RPPs involve developing relationships, the process is typically dynamic and lengthy.Partnerships require finesse and flexibility.Changes in systems may be incremental, may present stunning breakthroughs, or may move in fits and starts.It could be a combination of these trajectories.Thus, establishing and sustaining RPPs require patience and planning.Implementation science seeks to help us with the latter and assists us with realizing the need for the former (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005).
In our illustration, a grant application presented an opportunity for careful planning.We anticipated the need for different layers of partnership (i.e., leadership, research, and school-based teams), but we learned that additional layers were needed, with the addition of district and school administrators and site-based leaders of problem-solving leadership teams.We planned for liaisons between the district and school sites but needed a new plan when those liaisons were all reassigned to schools.We anticipated policies and procedures to provide a foundation for simply spreading the implementation of MTSS, but we learned that there was tremendous inconsistency in the commitment to, interpretation of, and execution of those policies and procedures.Partners who recognize the need for improved outcomes, who are open to change, who are respectful, and who follow through on their commitments exhibit the characteristics needed among researchers, practitioners, and community members alike.Despite relative stability among RPP members, absences from meetings occurred at times and the departure and replacement of two project coordinators affected the developing relationships and the execution of the RPP activities.We planned to disseminate our activities and results at participating schools, at the district level, and at conferences but encountered some resistance when barriers or unimpressive results were included.Despite the challenges encountered, our ability to fall back on our original plans and agreements served as a foundation and stabilizing force that reminded the RPP of its mission, goals, and objectives.
RPPs have the potential to bring together motivated individuals with varied and complementary skill sets.Consequently, partners' willingness to learn to communicate effectively given their different backgrounds, agendas, and jargon is a critical dimension of RPPs.Creative solutions to problems may be the ultimate product of the varying perspectives that researchers, practitioners, and community members bring to the table.Olswang and Goldstein (2017) characterize such research collaborations as "win-win situations."Researchers are rewarded if RPP findings are actually widely adopted in practice.Practitioners are rewarded if RPP findings that are implemented produce meaningful, positive outcomes for their clients.
RPPs formed around mutual respect and effective communication are needed to spur rigorous clinical practice research to determine how best to promote functional communication skills that lead to meaningful social, educational, vocational, and recreational life outcomes for our clients.We believe researchers and practitioners from communication sciences and disorders have a lot to contribute to participation, establishment, and promotion of RPPs.We encourage speech-language pathologists, audiologists, and researchers in the field to take advantage of these opportunities in their respective work settings.RPPs offer invaluable learning experiences for researchers, practitioners, and community stakeholders.Stressful interactions may occur on occasion due to differences in philosophies, belief systems, misconceptions, and perceived impositions due to the research design.However, effective teams can work together to anticipate and resolve these challenges.As Olswang and Goldstein (2017, p. 70) point out: True partnerships between researchers and practitioners and associated stakeholders are likely to be instructive, humbling, fun, and only occasionally frustrating.
When an RPP functions successfully, the impact of the research is increased because it involves direct contact and collaboration among those who are responsible for sustaining the innovations.RPPs have the potential to make our research count by being useful in real-world settings.Thus, RPPs offer a strategy for evidence-based practices to be developed, adopted, and implemented without the long delays typical of the traditional research to practice pipeline.
education/Voluntary Pre-K (VPK) blended classrooms and Head Start classrooms.Because of teacher attrition and nonparticipation, 42 teachers participated in Year 1.Of those 42, 23 teachers continued their involvement, plus 17 new teachers volunteered to participate, totaling 40 teachers at 23 sites in Year 2.
, supervisor of response to intervention (RtI)/multitiered system of supports (MTSS), and district research department staff.Most of the supervisors who oversaw the early childhood programs (Head Start, early childhood special education, and voluntary Pre-Kindergarten [Pre-K]) had not worked directly with the lead researcher in the past.However, they were familiar with his work in the community because they attended a presentation to share an innovative preschool language curriculum and results of an evaluation that was conducted locally and in two other states.They also learned about how this research fit into the ongoing work of the Center for Response to Intervention in Early Childhood funded by the IES.