Designing an approach to evaluation in a complex environment: How the evaluation of Ontario Health Teams adapted to a changing landscape

• Published in: International journal of integrated care 2022-05, Vol.22 (S2), p.167
• Main Authors: Embuldeniya, Gayathri, Hall, Ruth, McKellar, Kaileah, Wodchis, Walter
• Format: Article
• Language: English
• Subjects: Ethnography; Health care policy; Integrated delivery systems; Pandemics; Teams
• ISSN: 1568-4156; 1568-4156
• DOI: 10.5334/ijic.ICIC21255

IntroductionIn 2019, Ontario’s Ministry of Health launched Ontario Health Teams (OHTs), an ambitious initiative that seeks to ensure that all health care organizations, providers and patients across the province are connected to an integrated team with responsibility for eventually providing all healthcare services for a specific population. Our team was tasked with evaluating OHTs in the context of a landscape transformed by systemic restructuring, shifting healthcare policies and shaped by the COVID19 pandemic. We set out our approach to evaluation in response to this evolving landscape.Aims Objectives Theory or MethodsWe developed a two-phased approach to evaluation. The first formative phase was largely completed in March 2020, just before the pandemic reached Canada, providing contextual and baseline data on how OHTs came together, developed common vision, trusting relationships, governance, and a communication, enagagement and implementation plan. The second phase includes developmental evaluation to provide insight into the heterogeneity of integrated care approaches and the continuous evolution of new models of care. While the first phase provided a snapshot in time (largely January – March 2021), the second will provide ongoing monitoring, from March 2021 – February 2022.Highlights or Results or Key FindingsThe evaluation features five key attributes: longitudinality, ethnography, mixed-methods, incorporation of diverse perspectives, and co-design with OHT participants. The formative phase comprised 1167 surveys distributed across teams that submitted a full application to become an OHT and 125 in-depth interviews with 12 teams. The developmental phase focuses on 6 teams and involves interviews with diverse participants, ethnographic observations of planning meetings and participant journaling over time. Patient and provider experience surveys will also be conducted. The pandemic provided an opportunity to attend virtual meetings, enabling direct insight into team interaction and practice. The overlap of some OHTs across phases also enables insight into how the pandemic impacted OHTs and how OHTs evolved in response. Data collection was informed by key OHT participants who provided guidance and acted as a conduit between OHTs and the evaluation team. Vignettes of insights yielded by different methods will be shared.ConclusionsOur evaluation approach developed in response to the heterogeneity of integrated care approaches within a landscape shaped