SACME 2024 Annual Meeting Oral Abstract Presentations

1. Harnessing the Power of Natural Language Processing (NLP) in CPD Program Evaluation

Author(s) 

Kenya A. Costa-Dookhan, MSc, Medical Student, University of Toronto Temerty Faculty of Medicine

Marta Maslej, PhD, Staff Scientist, Krembil Centre for Neuroinformatics, Center for Addiction & Mental Health (CAMH)

Kayle Donner, MA, M.Ed., Research Methods Specialist, Center for Addiction and Mental Health (CAMH)

Faisal Islam, PhD, Manager, Education, Evaluation and Quality Improvement, Center for Addiction and Mental Health (CAMH)

Sanjeev Sockalingam, MD, MHPE, FRCP(C), FACLP, Chief Medical Officer, Vice-President Education, and Senior Scientist, Professor, Department of Psychiatry, CAMH and University of Toronto Department of Psychiatry

Anupam Thakur, MBBS, MD, MSc, Staff Psychiatrist, University of Toronto Department of Psychiatry, Education Scholar, CAMH

Background/context
With the increasing application of Natural Language Processing (NLP) in Medicine at large, medical educators are urged to gain an understanding and implement NLP techniques within their own education programs to improve the workflow and make significant and rapid improvements in their programs.

Purpose/thesis statement(s)
This project aims to provide essential tips inclusive of both conceptual and technical factors to facilitate the successful integration of NLP in medical education program evaluation.

Literature review/current perspective in the field
Recent literature (Chary et al 2019, Chan and Zary 2019, Masters 2020) illustrates artificial intelligence to be a valuable tool for rapidly analyzing large amounts of evaluation data. In particular, NLP techniques have the power to easily identify hidden patterns or trends in data that human evaluators might not be able to uncover and make predictions. Along with this, it offers the ability to provide instantaneous feedback, support decision making, and contribute to making significant improvements in medical education programs.

Theoretical framework(s)

Given that NLP use in medical education, including continuing professional development, is an emerging field, few primary studies were expected to support the tips developed. Collaborative discussion among education researchers and experts in NLP and ideas supported by peer-reviewed literature published in Ovid MEDLINE, Ovid Embase, PubMed, and Google scholar, contributed to the development of these tips. Furthermore, Kern’s six-step framework to curriculum design was used as a reference in development of the tips given the relevance of this framework in curriculum development, implementation, and evaluation broadly in health professions education.

Discussion
In summary, the following 12 essential tips can be applied to the integration of NLP in medical education program evaluation: 1. Decide if NLP addresses your education program need or question, 2. Build a team with expertise in NLP and data, 3. Set analysis outcomes that align with educational/curricular goals, 4. Consider the quality and quantity of data for NLP applications, 5. Explore the availability and utility of quantitative data (for informing or validating NLP), 6. Decide whether you will use NLP for patterns or prediction, 7. Find the right pre-processing approach for your data, 8. Explore different NLP methods for text representation and analysis, 9. Go beyond the default settings of your analysis to explore its sensitivity, 10. Be mindful of potential for algorithmic bias, 11. Have the appropriate security measures in place to protect student anonymity and privacy, and 12. Reflect on the use of NLP and disseminate evaluation findings. NLP applications in medical education hold high levels of promise to propel the effectiveness and efficiency at which feedback is provided and evaluation of programs are conducted.

Impact/relevance to the advancement of the field of CME/CPD
The 12 tips provided will assist current and future medical educators with using NLP for education program evaluation and assessing learner data to inform meaningful change for medical learners training at any stage in training. These tips will be widely applicable to CME/CPD educators, leaders and providers who are considering how NLP can be practically leveraged in the different phases of curriculum and education program design and delivery.

2. Exploring the Impact of ECHO Ontario Integrated Mental and Physical Health on Participants’ Approach and Attitude Towards the Care of Patients with Complex Needs

Author(s)

Sanjeev Sockalingam, MD, MHPE, FRCP(C), FACLP, Chief Medical Officer, Vice-President Education, and Senior Scientist, Professor, Department of Psychiatry, Center for Addiction and Mental Health (CAMH) and University of Toronto Department of Psychiatry

Victoria Bond, MSc, Acting Manager, Center for Addiction and Mental Health (CAMH)

Jake Chaput, BSc, Program Officer, Center for Addiction and Mental Health (CAMH)

Javed Alloo, MD, CFPC, MPL, Family Physician. Center for Addiction and Mental Health (CAMH)

Mark Bonta, MD, FRCP(C), Internal Medicine Specialist, Toronto General Hospital

Sophie Soklaridis, PhD, Scientific Director and Senior Scientist, Center for Addiction and Mental Health (CAMH)

Maria Zhang, RPh, BScPhm, PharmD, MSc, Pharmacist, Center for Addiction and Mental Health (CAMH)

Kathleen Sheehan, MD, DPhil, FRCPC, Psychiatrist, Toronto General Hospital

Background/purpose/inquiry question
The care and management of patients with multimorbidity, co-occurring physical and mental health concerns combined with psychosocial issues, is complex. Research demonstrates that these patients have higher mortality rates and poorer clinical outcomes. In Ontario, a high proportion of the care of patients with multimorbidity occurs within the primary care setting. However, many primary care providers (PCPs) supporting this patient population have limited training in these specialized areas and have poor access to specialist support, particularly in remote and under-serviced areas. As a response to these challenges, Project ECHO Ontario Mental Health launched the Integrated Mental and Physical Health (ECHO-IMPH) program.

Theoretical framework(s)
ECHO-IMPH is a tele-education model that builds capacity in the management of complex conditions by connecting PCPs throughout Ontario to specialist teams at academic health centers. Leveraging Moore’s outcomes framework, evaluation findings from ECHO-IMPH showed participants were highly satisfied with the program, and pre-post learning and confidence scores demonstrated statistically significant improvements. Building upon these findings, our team wished to examine the impact of the ECHO model on providers’ capacity to manage patient with multimorbidity after participation. Through post-cycle interviews, this research study aimed to understand how ECHO-IMPH influences participants’ clinical approach and attitudes toward patients with complex needs.

Methods
An exploratory qualitative approach was undertaken, with twenty-two (n=22) individual semi-structured interviews conducted following two cycles of ECHO-IMPH. We utilized a systematic thematic analysis following the Braun and Clarke six phase approach. A coding dictionary was developed, and transcripts were coded and analyzed in order to develop final themes.

Results/findings
Three major themes identified in the interviews were enhanced knowledge and skills, changes in attitude and approach, and space for reflection and exploration. Interviewees identified a shift in attitude towards patients with complex needs, from frustration to empathy with a more patient-centered lens. This helped facilitate a change in the practice of ECHO-IMPH participants towards patient-identified goals and functioning. The ECHO model created an environment that facilitated practice and attitude change through validation of challenges experienced and encouraging critical reflection of previous practice and how to apply new learning.

Discussion
Findings from this study indicate that ECHO-IMPH creates a safe community of practice that fosters practice change and attitude shifts towards complex patients in a primary care environment. Participants applied newly acquired knowledge and skills to provide more empathetic and patient-centered care for patients with complex needs. Based on the shift in perspectives described by participants, transformative learning theory was proposed as a model for how ECHO-IMPH created change in participants’ practice.

Limitations
This study included a small sample size and may not be fully representative of all participants. Our findings may be unique to ECHO-IMPH, and future research is needed to explore similar themes in other educational programs.

Impact/relevance to the advancement of the field of CME/CPD
This study demonstrates that ECHO-IMPH is an effective model for eliciting practice change in PCPs managing patients with complex needs. These findings can help educators facilitate meaningful change in their learners. Results highlight how knowledge and skills gained in CME programs such as ECHO-IMPH can improve interactions with patients who are conceptualized as complex.

3. Guiding the Needs Assessment Process from Application to Review: Research and Perspectives from a Canadian Continuing Professional Development (CPD) Office

Author(s)

Morag Paton, PhD, Associate Director, Maintenance of Certification & Education Consultation Services, CPD, Temerty Faculty of Medicine, University of Toronto

Trevor Cuddy, BCom, BA, MEd, Director, CPD, Temerty Faculty of Medicine, University of Toronto

Kate Hodgson, DVM, MHSc, CCMEP, Education Consultant, CPD, Temerty Faculty of Medicine, University of Toronto

Brien Wong, n/a, Lead Planner, CPD, Temerty Faculty of Medicine, University of Toronto

Christopher Chipman, BA, BEd, Business Development Officer, CPD, Temerty Faculty of Medicine, University of Toronto

Suzan Schneeweiss, MD, MEd, FRCPC, Associate Dean, CPD, Temerty Faculty of Medicine, University of Toronto

Purpose/problem statement
Writing rigorous needs assessments that incorporate perceived and unperceived needs is often a challenge for CPD accreditation applicants. CPD Planners and Program Directors report that summarizing needs is a time consuming and sometimes difficult or onerous task. Accreditation Reviewers acknowledge submitted information is sometimes incomplete or lacks detail. As part of an internal quality review process, our team sought to identify opportunities for improving the accreditation application process.

Methods/Approach(es)
We employed a descriptive analysis approach. After receiving an ethics exemption, we gathered baseline data of reviewer feedback on accreditation applications submitted to our office between 2020 and 2023. After data extraction and cleaning, we produced descriptive statistics (using SPSS) on reviewer scores for application components (including: final decision, target audience, financial, needs assessment, learning objectives, program design, and evaluation). We next analyzed two of these components, coding reviewer feedback in NVivo to identify possible target areas for change.

Findings
We analyzed 619 programs, identifying that the needs assessment and learning objectives components were strong contributors to programs requiring minor or major revisions upon review. We further identified that missing unperceived needs and providing non-measurable/observable learning objectives were primary reasons for applications requiring further revision. To address these gaps, we adapted an existing needs assessment template originally designed by a CPD planner to guide faculty through submission requirements. The new Needs Assessment Guide aims to support Program Directors, Scientific Planning Committees, Planners and Applicants through the needs assessment development process and facilitate Reviewers in their assessments. It provides a concise form of reporting perceived and unperceived needs, program goals, and learning objectives while clarifying Canadian accreditation requirements (CFPC/RCPSC) to adequately source data.

Discussion
To date, CPD community members are appreciative of efforts to conduct research on internal accreditation processes designed to simplify the process and strengthen the quality of the submitted programs. Early feedback on the Guide has been positive with CPD faculty leaders indicating their support for the tool. After consultation with planners, the guide has been further revised and simplified, and will shortly be deployed for internal use.

Barriers/facilitations
A facilitator for this work is collaboration between staff and faculty and joint efforts to improve processes and programs. Barriers include the need to balance simplicity of the tool design with the rigor required for the submitted material. We anticipate that the tool will help streamline planner-SPC communications, and attention will be needed to introduce faculty to the new Guide.

Impact/relevance to the advancement of the field of CME/CPD
This quality improvement project is evidence that conducting research with professional services staff and faculty on operational processes can promote best practices in accreditation. Although specific accreditation requirements differ between jurisdictions, educating planning committees on the use of multiple data sources to identify perceived and unperceived needs, and demonstrating links between needs and program goals and objectives is a common goal and may strengthen the effectiveness of CPD programming to achieve practice change.

4. A Framework for Integrating Structural Competency into Physician Leadership Curricula

Author(s)

Branka Agic, MD, PhD, Scientist, Education, Center for Addiction and Mental Health

Akwatu Khenti, PhD, Director, Community Resources, City of Toronto

Anna MacLeod, PhD, Professor and Director of Education Research, Dalhousie University, Executive Director, Across Boundaries

Ayelet Kuper, MD, DPhil, FRCPC, Physician, University of Toronto

Constance H. LeBlanc, MD, FCFP(EM), MAEd, CCPE, MBA, Professor, Department of Emergency Medicine, Dalhousie University

Cynthia Whitehead, MD PhD, Director and Scientist, University Health Network/University of Toronto

Elizabeth Lin, PhD, Scientist, Education, Center for Addiction and Mental Health (CAMH)

Howard Fruitman, MA, MEd, Instructional Designer, Simulation and Digital Innovation, Center for Addiction and Mental Health (CAMH)

Jamiu Busari, MD, MHPE, PhD, CCPE. Consultant Pediatrician & Dean, HOH Academy, Horacio Oduber Hospital

Morag Paton, PhD, Associate Director, Maintenance of Certification & Education Consultation Services, CPD, Temerty Faculty of Medicine, University of Toronto

Quincy Norman. Vaz, MBBS. Research Analyst, Centre for Addiction and Mental Health (CAMH)

Rabia Zaheer, MPH, Research Coordinator, Centre for Addiction and Mental Health (CAMH)

Rowen Shier, MA, Research Analyst, Centre for Addiction and Mental Health (CAMH)

Sandy Buchman, MD CCFP (PC) FCFP, Freeman Family Chair in Palliative Care, North York General Hospital

Sanjeev Sockalingam, MD, MHPE, FRCP(C), FACLP, Chief Medical Officer, Vice-President Education, and Senior Scientist, Professor, Department of Psychiatry, CAMH and University of Toronto Department of Psychiatry

Yuliya Knyahnytska, MD, PhD, Clinician Scientist, Staff Psychiatrist, CAMH, University of Toronto, Queen’s University

Sophie Soklaridis, PhD, Scientific Director and Senior Scientist, Centre for Addiction and Mental Health

Background/purpose/inquiry question
Evidence shows that reducing inequities in access and care outcomes for diverse populations requires greater diversity in medical leadership. One of the main strategies for increasing leadership diversity and addressing systemic inequities has been to provide leadership training on equity, diversity and inclusion (EDI). However, a 2021 environmental scan has found a dearth of nuanced EDI content in physician leadership programs in Canada and the US. As a way forward, we suggest embedding structural competency into physician leadership curricula using an evidence-based framework to prepare physicians to identify and address the structural factors that perpetuate inequities in medicine.

Theoretical framework(s)
The framework is informed by the Health Equity and Inclusion Framework for Education and Training, a Compassionate Leadership approach and Transformative Learning Theory. Principles of equity, diversity, inclusion, anti-oppression and anti-racism (EDIA) form the core of the framework. The framework aligns with the CanMEDS physician competency framework, specifically with the competencies for the Leader role, and the Canadian Medical Association’s Policy on Equity and Diversity in Medicine.

Methods
The framework is based on a review of literature on structural competency and key competency frameworks and guiding documents for physician leaders. An advisory group composed of individuals who have demonstrated leadership in EDIA, knowledge of underrepresented and marginalized groups, and/or experience with physician leadership programs guided the development process.

Results/findings
As leaders, physicians are expected to take responsibility for the delivery of high-quality care and transformation of the health system through their roles as clinicians, administrators, scholars and teachers. Building on Metzl and Hansen’s 2014 definition, we define structural competency for physician leaders as the knowledge, skills and attitudes required to challenge inequities and lead structural change in the clinical workplace, medical education, research and academia. We have identified nine enabling competencies essential to achieving structural competency: 1. Recognize and reflect on personal biases, assumptions and perceptions 2. Challenge policies and practices that create inequities in education, workplace, academia and health care 3. Integrate EDIA in teaching, training and mentoring 4. Promote recruitment, hiring, promotion and cultivation of historically underrepresented groups 5. Support research, collection and use of data related to EDIA 6. Enhance equity in access, quality and outcomes of care 7. Allocate resources to improve equity 8. Build partnerships with organizations and leaders representing marginalized groups 9. Demonstrate inclusive and compassionate leadership.

Discussion
This framework provides practical examples, learning activities and resources to support each of the nine competencies identified through the review, research team and advisory group. Depending on the scope and goals of the physician leadership program, educators can incorporate one, a few or all of the enabling competencies.

Limitations
The framework aligns with the CanMEDS framework and may need to be adapted for other physician competency frameworks outside of Canada and US.

Impact/relevance to the advancement of the field of CME/CPD
This framework is a practical and easily applied guide on integrating structural competency into physician leadership curricula to build their capacity required to recognize and challenge inequities in all aspects of their work.

5. Implications of the ABMS Member Boards Becoming the Largest CME Provider in the U.S.

Author

R. Van. Harrison, PhD, Professor Emeritus, University of Michigan Medical School

Background/context
The 24 Member Boards of the American Board of Medical Specialties are together becoming the largest provider of continuing medical education in the U.S. The Boards certify approximately 70% of the MD and DO physicians in the U.S. The Boards are evolving from performing assessments about every 10 years to using machine-based learning to assess and educate physicians throughout each year.

Purpose/thesis statement(s)
Individuals and organizations providing CME/CPD need to understand the expanding educational role of the Boards and implications for their CME programs.

Literature review/current perspective in the field
Lyness JM, McMahon GT. The role of specialty certification in career-long competence. Academic Medicine ():10.1097/ACM.0000000000005314, July 04, 2023. | DOI: 10.1097/ACM.0000000000005314

Theoretical framework
Frameworks for translating knowledge into practice help clarify the role of the Boards’ expansion into continuing education. Pathman et al (1996) provide a useful simple model: Aware/Agree/Adopt/Adhere.

Discussion
Each Board outlines what physicians in that specialty are expected to know and do. Boards are using advances in technology to offer ongoing machine-guided assessment and learning. At intervals during the year, items are provided online for physicians to complete when and where convenient. Computer assisted instruction provides items, immediate feedback on responses, explanations of answers, access to additional educational information, and retesting to confirm learning occurred. Items are updated to reflect evolving medical knowledge.

Boards also award credit for participation in knowledge self-assessment (component of old “Part II” MOC). Participation demonstrably advances learning while minimizing inconvenience and expense. Certified physicians have responded very positively to this new approach to assessment and learning. Regarding translating knowledge into practice, the ongoing systematic review of knowledge by individual physicians clearly advances an individual’s general “Awareness” of and “Agreement” with current knowledge. To the extent an individual controls the practice environment, the review may advance “Adopting” a practice and “Adhering” to it.  

At a practical level, physicians participating in systematic knowledge review may consider participation in other CME activities to be less important. Physicians’ time is already filled with competing demands. The benefits of systematic review, feedback, and personal learning are immediate and obvious. Physicians often do not consider distinctions between types and circumstances of learning, e.g., “just in case” vs. “just in time,” “sampling a range of knowledge” vs. “focused review of a specific medical condition/issue,” “general knowledge” vs. “knowledge about local implementation,” and “individual learning for knowledge” vs. “group learning as an initial step in organizational change.” With Boards providing knowledge self-assessment credit, interest in this credit through other CME providers will decrease.

Impact/relevance to the advancement of the field of CME/CPD
CME providers need to understand the emergence of widespread systematic computer-assisted assessment and learning for certified physicians and its implications. CME providers must: • Learn about changes by specialty board and consider implications for educational needs. • Review the types and circumstances of CME activities they provide. • Adjust their CME activities to better meet needs for CME in this evolving environment. • Communicate how their CME activities compliment/build on new learning activities of Boards. • Consider how to address learning needs of non-certified physicians. These implications will be elaborated and addressed.

6. The Effect of Online, Spaced Repetition of Content-Rich Multiple-Choice Questions on Long Term Knowledge Retention and Transfer among Family Physicians

Author(s)

David Price, MD, FAAFP, FACEHP, FSACME, Professor Family Medicine/Sr. Advisor to the President/CPD Coach and Consultant, American Board of Family Medicine/University of CO Anschutz School of Medicine

Ting Wang, PhD, Senior Psychometrician, American Board of Family Medicine

Thomas O’Neill, PhD, President and CEO, American Board of Family Medicine

Warren Newton, MD, MPH, President and CEO, American Board of Family Medicine

Background/purpose/inquiry question
The American Board of Family Medicine previously demonstrated that spaced repetition delivered through rigorously developed, case-based, multiple-choice questions (MCQ) delivered quarterly through an online platform enhances knowledge retention at 18 months in a large cohort of practicing family physicians. This follow-up analysis evaluates the effect of 5 different spaced repetition strategies from the knowledge retention study on transfer (generalizability) of knowledge using MCQs with different clinical scenarios than he previously repeated questions.

Theoretical framework(s)
Spaced repetition, testing effect.

Methods
Physicians completing the ABFM Continuing Knowledge Self-Assessment (CKSA) in the 4th quarter of 2020 were randomized to a control group or one of 5 spaced repetition conditions over the subsequent 5 calendar quarters. Control group participants received no repeated questions. Participants in the other 5 groups received 6 questions repeated either once or twice; the interval between repetitions differed between each group. Confidently but incorrectly answered baseline questions were prioritized for repetition with decreasing priority for those questions answered with lesser degrees of confidence and those answered correctly but not confidently. In quarter 6 of the study, all remaining participants received their repeated questions. In quarter 10 of the study (April – June 2023) remaining participants received “cloned” questions; these questions presented a different clinical scenario than the original question but emphasized the same clinical concepts. The primary analysis compared differences in knowledge transfer (as measured by the percentage of cloned questions answered correctly) between physicians receiving any spaced repetition questions in quarters 1-5 compared with the control group who will have had initially gone 18 months without seeing repeated questions. Subgroup analyses compared differences in transfer between physicians who initially received one repetition compared with those initially receiving two, differences in transfer between the two original single spaced repletion strategies, and differences in transfer between the three original double spaced repetition strategies.

Results/findings
As was the case after quarter 6, the groups initially receiving spaced repetitions performed better on the transfer questions than those who did not initially receive spaced repetition questions; those in the initial two spaced repetition groups performed better than those who initially received one spaced repetition. There were no substantive differences between the original one spaced repetition strategies and between the original two spaced repetition strategies. Less, but substantial amounts of transfer occurred even in the control group (who due to the study design essentially had one spaced repetition four quarters earlier).

Discussion
Online spaced repetition using clinical scenario based MCQs is feasible over a long period of time in large cohorts of practicing physicians and leads to improved knowledge transfer as well as improved knowledge retention.

Limitations
The study design did not include participants who had never seen a repeated question. Other health professionals and physicians in specialties other than family medicine were not included. Workplace impact was not assessed.

Impact/relevance to the advancement of the field of CME/CPD
This study can inform the development and utilization of spaced repetition strategies for assessing both knowledge retention and assessing knowledge transfer in health professions CPD.

7. Expanding our Evaluations of Bias: A Pilot Study of Four Canadian University CPD Providers

Author(s)

Morag Paton, PhD, Associate Director, Maintenance of Certification & Education Consultation Services, CPD, Temerty Faculty of Medicine, University of Toronto

Clare Cook, PhD, Research Coordinator, Continuing Education and Professional Development, NOSM University

Tanya Hill, MSc, Evaluation Specialist, Continuing Professional Development, Dalhousie University

Background/context/purpose
CPD has traditionally focused on monitoring financial bias in accredited programs but can fail to recognize other types of bias with potential unintended and negative impacts on patient care. Canadian accreditors, who currently mandate a standardized bias question, have signalled intent to reconsider how bias is evaluated as one way to help address inequities in healthcare. This descriptive study evaluated how expanding the general bias question and prompting for specific types of bias may influence the reporting of bias in continuing professional development for physicians.

Theoretical framework(s)
Social constructivism and critical theory inform this research.

Methods
Four Canadian university CPD offices piloted an expanded evaluation approach for reports of bias. Following ethical approval/exemption, each institution adapted the standardized bias question to list specific examples of potential bias, including speakers’ funding, mention of pharmaceuticals or products, and personal opinions. The language, format, and process of deploying the expanded bias question differed by university because of varying contexts, tools, and processes. Expanded bias questions were deployed between 2021 and Feb 2023 within a cross-section of accredited programs incorporating a range of topics, target audiences, credit type, and CanMEDs roles. Data about bias reporting frequency, reports of bias within sessions, and open-text descriptions of bias were collected in Qualtrics, Opinio, and SliDo and analyzed in Excel and/or SPSS.

Results/findings
Expanded bias questions were piloted on 23 programs (188 sessions) across 4 sites​ with total registrant numbers of 3601​. Response rates to the bias question ranged from 12% to 91.67%. At the session level, 64 of 188 sessions (34%) were reported to have some kind of bias. However, reports of bias at the respondent level were low for each of the four sites, with only 1.27%, 1.32%, 1.93% and 2.93% of respondents reporting any type of bias. Content analysis of respondent comments displayed a nuanced understanding of bias, including concern with a lack of supporting evidence and reliance on opinion or experience rather than best practice. Others noted that the bias observed was appropriate and/or reasonably mitigated by the speaker.

Discussion
Expanding the bias question to include specific examples of bias provides more information to evaluators on potential biases within programs, however, individual reports of bias remain low. Aiming for “100% bias-free programming” (which is a declared outcome or goal of some programs) may not be as desirable a goal as aiming for a fulsome reporting of multiple forms of bias. The low rates, and high variance, in reported bias, raise questions and support further research about types and appropriateness of bias, learner perceptions of bias in CPD, factors affecting motivations for reporting, and the cognitive load of learners.

Impact/relevance to the advancement of the field of CME/CPD
This study identified that institutional evaluation practices may influence the reporting of bias (or types of bias) in CPD programs signaling the potential of missing forms of bias that may ultimately impact patient care and health equity. This study contributes to efforts to improve evaluation design and reporting practices for bias in CPD programs.

8. Strengthening the Position of Continuing Education as a Foundational Element of Quality Improvement

Author(s)

Joanne Goldman, PhD, Education Scientist, University of Toronto

Lisha Lo, MPS, Research and Education Coordinator, Centre for Quality Improvement and Patient Safety, University of Toronto

Sanjeev Sockalingam, MD, MHPE, FRCP(C), FACLP, Chief Medical Officer, Vice-President Education, and Senior Scientist, Professor, Department of Psychiatry, Centre for Addiction and Mental Health and University of Toronto Department of Psychiatry

Stella Ng, PhD, Education Scientist, Centre for Advancing Collaborative Healthcare and Education, University Health Network

Ryan Brydges, PhD, Education Scientist, Unity Health Toronto

Tara Burra, MA, MD, FRCPC, Physician, Education lead, Sinai Health

Julie La, MD, Physician, PhD student, Queens University

Brian Wong, MD, FRCPC, Physician, Associate professor, Director, Centre for Quality Improvement and Patient Safety and Sunnybrook Health Sciences Centre and Department of Medicine

Work in progress

Background/context/inquiry question
To date, efforts to integrate the fields of continuing education (CE) and quality improvement (QI) have primarily focused on CE about QI. Another promising approach that has received less attention involves strengthening the rigour and presence of CE within QI initiatives, where the aim is for a robust CE component of a broader systems-based QI intervention. This study addresses this gap through a scoping review of CE within QI initiatives and a qualitative interview-based study with CE and/or QI leads of these studies.

Reference to current literature/perspective on the topic
Despite the existence of examples in the literature of QI initiatives where CE plays a prominent role, rarely are education activities within QI explicitly labeled or viewed as ‘CE’. Therefore, education may fail to achieve its intended impact because QI teams have not optimized its development and implementation with what is known from education research and theory prior to its deployment, reinforcing the perception of education as a ‘less effective intervention’. There is a need to make more explicit the ways that CE is positioned in relation to other systems-based interventions to ensure that education activities are synergistic and complementary.

Theoretical framework(s)
Six common paradigms of education in health professions education (behaviorist, cognitivist, constructivist, sociocultural, humanist, and transformative) will provide a framework for analyzing the studies included in the review. Billett’s theory of workplace learning and specifically his conceptual framework of affordances will be used in the qualitative study to describe factors that influence QI teams’ use of CE in their QI initiatives

Methods
We are conducting a scoping review of the QI literature to map out the varied purposes that CE fulfills as part of QI initiatives and how QI teams draw on CE evidence and theory when developing and delivering the education components of their QI initiatives. We will then identify authors of the studies from the scoping review who used CE in their QI work and make explicit mention of CE or education evidence and conceptual or theoretical frameworks and invite them to participate in a semi-structured interview. We will conduct an interpretive thematic analysis using both an inductive approach, where analysis is derived directly from the data in relation to our research questions, and a deductive approach, where analysis is informed by Billett’s conceptual framework of affordances.

Impact/relevance to the advancement of the field of CME/CPD
Study findings will inform more deliberate use of CE within QI by highlighting the importance of clarifying the intended purposes for including education in a QI initiative and helping to select recommended CE approaches informed by evidence and theory to optimize the education components. Such incorporation of CE within broader QI initiatives will help to move the CE field beyond the classroom and conferences to the workplace, expand beyond learner outcomes to patient and health-system-level impacts, and be data-driven and integrated within systems of patient care.

Preliminary Findings (if any)
This presentation will report on the scoping review of this study. Our search of 21 quality and safety journals and the Journal of Continuing Education in the Health Professions produced 1674 papers. We plan to have the scoping review completed by March 2024.

9. A Lifeline for Uncertainties: How CPD Can Foster Organizational Resilience

Author(s)

Sophie Soklaridis, PhD, Senior Director and Senior Scientist, Center for Addiction and Mental Health

Rowen Shier, MA, Research Analyst

Centre for Addiction and Mental Health (CAMH)

Rabia Zaheer, MPH, Research Coordinator, Centre for Addiction and Mental Health (CAMH)

Michelle Scully, BA, Research Analyst, Centre for Addiction and Mental Health (CAMH)

Betsy W. Williams, PhD MPH FSACME, Clinical Program Director, Professional Renewal Center

Sam J. Daniel, MDCM, FRCSC, CPC(HC), Professor and Chair, Department of Pediatric Surgery, McGill University

Linda Dang, BA, Research Analyst, Centre for Addiction and Mental Health | CAMH

Sanjeev Sockalingam, MD, MHPE, FRCP(C), FACLP, Chief Medical Officer, Vice-President Education, and Senior Scientist, Professor, Department of Psychiatry, CAMH and University of Toronto Department of Psychiatry

Martin Tremblay, PhD, Research and Innovation Advisor, Fédération des Médecins Spécialistes du Québec

Background/purpose/inquiry question
During the pandemic, the field of CPD faced unprecedented challenges. The emergency restrictions and closures, coupled with the pressing need to disseminate rapid information about the virus, created a pivotal moment for the field to demonstrate its endurance and value. Moreover, the pandemic exacerbated issues of clinician burnout, EDI and health inequities which emerged as priority areas of concern. It is imperative to understand how CPD leaders and organizations responded to these challenges in order to extract key takeaway lessons and better inform organizational resilience within CPD.

Theoretical framework(s)
In this qualitative study, we utilized inductive thematic analysis to interpret patterns of meaning in participants’ experiences of the impact of COVID-19 on CPD. We embraced a broad transformative justice lens in later stages of analysis to identify potential avenues for CPD to meaningfully address complex challenges and effectively implement change.

Methods
A qualitative design was chosen to elicit rich in-depth data that captures the insights, experiences and perspectives of CPD leaders. Twenty-three participants were recruited via purposive and convenience sampling from Canada and the USA. Virtual semi-structured interviews took place between April to September 2022 and lasted approximately 60 minutes. Interviews were audio-recorded, transcribed, de-identified and analyzed thematically.

Results/findings
The following strategies to embed organizational resilience into CPD were endorsed by CPD leader participants or inspired by their comments: 1. Advocate for the recognition of clinicians’ humanity; 2. Move beyond individual-level solutions towards systemic and organizational interventions to address the root causes of burnout, health inequities and structural oppression; 3. Develop concrete strategies to help leaders integrate EDI into CPD curriculum; 4. Increase emergency preparedness through a) promoting a culture of innovation, b) facilitating interprofessional and patient collaboration, and c) fostering growth mindset and adaptive learning skills in CPD education.

Discussion
Our findings indicate that top-down and system-level change is required to support the resilience of CPD members and the field more broadly. This is particularly relevant as clinicians continue to grapple with key issues including burnout and health inequities, requiring ongoing support from CPD leadership. Finally, this study highlights key lessons learned from the COVID-19 pandemic that CPD organizations can apply to overcome, adapt and adjust to future exigent conditions.

Limitations
We note that although purposive and convenience sampling facilitated the recruitment of diverse participants with expert knowledge in relation to our research question(s), these approaches may increase risk of bias.

Impact/relevance to the advancement of the field of CME/CPD
The implementation of the above strategies may improve the field’s ability to respond to ongoing social emergencies, including burnout, health inequities and structural oppression. Furthermore, these strategies highlight the importance of emergency preparedness in anticipation and preparation for future crises. Finally, these hold the potential to elucidate the value of CPD in medicine and beyond.

10. Exploring Extended Reality (XR) Applications in Continuing Professional Development

Author

Vernon Curran, PhD, Associate Dean of Educational Development, Memorial University

Purpose/problem statement
Extended reality (XR) has emerged as an innovative simulation-based learning modality. XR is an umbrella term referring to all immersive technologies including virtual reality (VR), augmented reality (AR), mixed reality (MR), and other computer-generated realities using head-mounted displays (HMDs). The purpose of this presentation is to describe the emerging evidence around the use of XR technologies in medical education and CPD, and describe ongoing work investigating the effectiveness and potentials of XR technologies for CPD delivery.

Methods/Approach(es)
An integrative review of the peer-reviewed literature was undertaken to explore the nature of the evidence, usage, and effectiveness of XR modalities across the medical education continuum, including CPD. A phenomenological qualitative study was conducted to explore healthcare provider perceptions and experiences with using 360 degree video and VR headsets for CPD, and a randomized control trial (RCT) was used to compare the efficacy of 360 degree video with computer-generated VR for CPD. An exploratory study is underway to examine healthcare providers’ perspectives of potential uses of VR across a variety of disciplinary, knowledge and skill domains.

Findings
The emerging evidence suggests that different forms of XR technology applications have potential in creating immersive learning experiences that are engaging and lead to learning outcomes that appear to be equivalent to, or in some areas potentially more effective than, traditional methods for teaching and learning in medicine. Opportunities for adopting and using XR technologies in diverse fields of medical education are numerous, and in some knowledge and skill training areas, XR platforms may offer more cost-effective means for curriculum delivery.

Discussion
XR technologies offer greater portability with no heavy manikin parts to transport, repair, or safeguard, and no consumable parts that require replacing. XR may also provide greater standardization, replicability in experience, and accessibility. It can be distributed widely and has the potential to have a broader impact through increased learner engagement and improved spatial representation and learning contextualization.

Barriers/facilitations
As an emerging simulation-based learning technology, there is still much to learn about best approaches to adopting and using XR across different CPD formats, and for which knowledge and skill domains it is best suited. Further research to explore the effectiveness of XR use in CPD is needed, including more rigorous evaluation studies that compare different types of XR systems along with traditional teaching approaches.

Impact/relevance to the advancement of the field of CME/CPD
Different forms of XR technology applications have potential in creating immersive learning experiences that are engaging and lead to learning outcomes that appear to be equivalent to, or in some areas potentially more effective than, traditional methods for teaching and learning. XR platforms may offer more cost-effective means for CPD program delivery when compared to other simulation-based medical education modalities as well as greater portability and access to CPD, particularly for rural and remote healthcare providers in areas of knowledge and skill updates, refreshers and/or booster training.

11. Co-producing Health Education Research: Working Equitably in a Hierarchical System

Author(s)

Holly Harris, BA (Hons), MA, Research Coordinator, Centre for Addiction and Mental Health (CAMH)

Gail Bellissimo, Program Engagement Co-Facilitator, Centre for Addiction and Mental Health

Anna Di Giandomenico, BA, Lived Experience Partner, Centre for Addiction and Mental Health

Rowen Shier, MA, Research Analyst, Centre for Addiction and Mental Health

Elizabeth Lin, PhD, Scientist, Centre for Addiction and Mental Health

Sam Gruszecki, CPS, Coordinator, Collaborative Learning College, CAMH

Jordana Rovet, M.S.W., R.S.W., Coordinator, Centre for Addiction & Mental Health

Georgia Black, BA, Research Analyst, Centre for Addiction and Mental Health

Sophie Soklaridis, PhD, Scientific Director and Senior Scientist, Centre for Addiction and Mental Health

Purpose/problem statement
Co-production, in a healthcare context, is a process where people with lived experience (PWLE) of health system encounters are recognized as experts and collaborate alongside people with professional and/or academic expertise in the design, actualization, and dissemination of education and research. In this presentation, we explore a Canadian case example that illustrates the ways an education research team, situated in a healthcare facility, embodied the principles of co-production. We argue that co-producing health education, such as continuing professional development (CPD) initiatives, and research with those who access services is necessary, not only to create impactful and relevant results but more importantly for ethical engagement, equity, and inclusion.

Approach(es)
We will describe how our team created relational conditions that allowed for the inclusion and validation of all members’ voices, experiences, and expertise. Specifically, we will explore the nuances of co-production; benefits, wicked questions, challenges, and lessons. We will outline key principles that we embraced to navigate power and foster meaningful collaborations among people with diverse perspectives. Furthermore, we will explore how these principles can be enacted in CPD.

Findings
We identified four key principles of our co-production process: 1. Power sharing: Power is deeply ingrained in the systems and structures in which we operate. In contrast to traditional power structures, our team employed distributive leadership approaches. 2. Multi-directional learning: In our project, multi-directional learning involved each team member simultaneously assuming the role of teacher and learner, co-creating new knowledge at the intersection of diverse perspectives. 3. Connection through vulnerability: Our team challenged traditional power dynamics and centered lived experience by putting our “professional hats” aside and sharing from a place of vulnerability. 4. Slow and steady wins the race: Unlike ‘traditional’ approaches that prioritize rigid timelines, strict deadlines, and productivity outputs, we embraced flexibility and allowed time for processes, structures, and relationships to develop.

Discussion
We will invite participants to reflect on what equitable partnerships entail and explore how the principles presented may significantly enrich and shape their collaborative endeavors. We will issue a call to action for those in traditional positions of power to reflect on their positionality, make space for voices that have been historically marginalized, embody empathy and compassion, and explore the transformative value of equitable partnerships between those with learned and lived expertise.

Barriers/facilitations
Co-production and equitable partnerships are a journey rather than a destination. We will encourage audience members to start where they are and recognize that the pursuit of equitable partnerships is a political act of system transformation. By approaching this work with a growth mindset, a commitment to ongoing learning, and accountability, we can realize a more equitable and inclusive future in which PWLE, those with expertise from academia and profession, and people who have both of these perspectives have equal seats at the table.

Impact/relevance to the advancement of the field of CME/CPD
Engaging PWLE throughout the design and actualization of continuing professional development initiatives can have transformative systemic impacts. This involvement is crucial for infusing CPD initiatives with real-world insights on health and health systems navigation facilitating approaches that reflect compassionate, empathetic, and humanistic care.

12. Addressing the Health Impact of Climate Change Through the Lens of Environmental Philosophy

Author(s)

Nasim Gheshlaghi Azar, PhD, Lecturer in Medical Education, Medical Education Department, School of Medical Education & Learning Technologies, Shahid Beheshti University of Medical Sciences

Mohammad Reza, PhD, Water and Waste Water Manager, Omrab Engineering Company

Background/context
Climate change has been declared as a global health threat of the 21st century (1,2). Regarding the urgent need and responsibility for health professionals to address the health impacts of climate change (3), CME/CPD has an essential role in empowering them as the change advocates.

Purpose/thesis statement(s)
The purpose of this perspective is to present the ways that CME/CPD can empower the health professionals to act not only as physicians, but also as change advocates, in order to address the health impacts of climate changes.

Literature review/current perspective in the field
Health professionals are confronted with the significant impact of climate change on human health. As physicians, they must be able to recognize, prevent and treat climate-induced diseases (4). Furthermore, it is part of a physician’s duties to inform patients and the public about health consequences of climate change (5,6,7,8). Additionally, their trusted social status empowers them to become societal role models and leaders in addressing climate change (9), to help initiate change in business, politics, and education (10). Educating health professionals, in order to prepare them for these roles, is therefore indispensable (11).

Theoretical Framework
The theoretical framework of this perspective is based on the “Environmental philosophy”, which raises crucial questions about human- environment relations, environmental challenges such as environmental degradation, pollution and climate change, and environmental justice (12,13,14).

Discussion
Despite the importance of health professionals’ responsibility in relation to the health impact of climate change (15,16), their knowledge about this issue seems to be incomplete. Lack of knowledge is notably perceived as a barrier in communicating with the patients and public about the health consequences of climate change (16). In particular, the central role of physicians, as “eco-ethical” leaders in supporting a societal transformation in order to establish the culture of adaptation and mitigation in the context of climate change is inevitable (9). However, the low level of willingness of today’s medical students to act as social role models, must be critically questioned whether this lack of willingness is due to an ignorance of the health effects of climate change or whether this responsibility is seen to lie with other professions, which makes it difficult to make a statement specifically related to health professions (3). In spite of the need for incorporating content of climate change in the medical curriculum, just under 15% of the universities have already made this integration (17). Furthermore, many of today’s doctors are trained in an era when the health effects of climate change were not considered as a crucial requirement of health services. In this way, CME/CPD plays a vital role in urgently educating the physicians as the change advocators to acquire the knowledge, skills and attitude needed for addressing the health impacts of climate change.

Impact/relevance to the advancement of the field of CME/CPD
Incorporating an “eco-health” course in the CME/CPD program is a necessity for addressing the health impacts of climate change. Some of the key subjects of this course are as following: -Direct and indirect health impacts of climate change (18,19) -Human-ecosystem relationships (man-made climate change) (20,21) – Social dynamics in relation to environmental justice (10) -Climate-associated diseases and shifting disease patterns (1,19) -Adaptation and mitigation strategies for more sustainable and resilient management and delivery of health services (11) -Social responsibility of physicians as an informative, educational, and change agents to people (3) -Critical thinking as an important overarching skill that enables reflection on problems and their solutions (20).

References (Available upon request from author.)

13. Impact of CPD Courses Format on Medical Specialists’ Behavioral Intention: A Comparative Study

Author(s)

Martin Tremblay, PhD, Research and Innovation Advisor, Fédération des Médecins Spécialistes du Québec

Gloria Ayivi-Vinz, MSc, Student, Université Laval

Felly Bakwa, MSc, Student, Université Laval

Sam Daniel, MD, Director, Fédération des Médecins Spécialistes du Québec

Georgina Dofara, MSc, Research professional, Université Laval

Souleymane Gadio, PhD, Research professional, Université Laval

France Légaré, MD, Professor, Université Laval

Denis Talbot, PhD, Professor, Université Laval

Background/purpose/inquiry question
Many organizations developed virtual CPD activities to meet physicians’ needs during the COVID-19 pandemic. It is well established that virtual activities are as effective as in-person activities with respect to knowledge acquisition. However, little is known about the difference in behavioral intention between these two formats. The aim of this study was to evaluate the impact of CPD course format (in-person or online) on medical specialists’ behavioral intention in Quebec, Canada.

Theoretical framework
This study was informed by Godin’s integrated conceptual framework of socio-cognitive factors influencing behavioral intention of healthcare professionals.

Methods
We conducted a comparison between two pre-post intervention studies. The first study comprised medical specialists who participated in one of nine pre-determined in-person CPD courses offered at the FMSQ 2019 Annual Meeting. The second study comprised medical specialists who participated in one of these nine courses, which were adapted into an asynchronous virtual format made available between 2020 and 2022 on our learning management system. Data was collected before (T0) and after each course (T1) using the CPD-REACTION questionnaire which assessed the behavioral intention targeted by each course (primary outcome) and four behavioral psychosocial determinants (beliefs about capabilities, moral norm, social norm, and beliefs about consequences).

Results/findings
Overall, 164 physicians participated in the in-person courses and 636 in the online courses. The average behavioral intention was 5.99 ± 1.31 for the in-person format and 5.53 ± 1.62 for the online format at T0. The average behavioral intention was 6.43 ± 0.80 for the in-person format and 5.98 ± 1.40 for the online format at T1. The mean difference in behavioral intention between T0 and T1 was 0.44 (CI: 0.15-0.74; p=0.003) in the in-person study and 0.45 (CI: 0.29-0.59; p< 0.0001) in the online study. However, there was no statistical difference in behavioral intention between these two studies (p=0.31). Multivariate analysis showed that beliefs about capabilities (β=0.15, p=0.001), moral norm (β=0.75, p< 0.0001), and beliefs about consequences (β=0.11, p=0.04) influenced post-course intention for the in-person study and (β=0.50, p< 0.0001), (β=0.40, p< 0.0001), and (β=0.22, p< 0.0001) for the online format, respectively.

Discussion
While both studies increased behavioral intention, we observed no impact of the format on the primary outcome of interest. These results are in line with current knowledge regarding the impact of CPD activities’ format. Based on multivariate analysis results, CPD organizers could consider integrating related content that could increase ethical acceptability, confidence about adopting a specific behavior, and reinforce the perception that the behavior would be useful and beneficial, to further increase behavioral intention.

Limitations
The main limitations are derived from the pragmatic nature of the study design: time difference between both studies, varying T0-T1 intervals during data collection for the online format, and how the course content was adapted for an online format.

Impact/relevance to the advancement of the field of CME/CPD
Social-cognitive theories pertaining to behavioral change provide an appropriate theoretical framework to assess the impact of CPD activities. As behavioral intention is a proxy of healthcare professionals’ behavior, a better understanding of the factors influencing intention could support the development of more effective educational activities.

14. Performance Improvement Continuing Medical Education as an Intervention to Improve Physician Professionalism

Author(s)

Caitlin Hurley, MD, CME Medical Director, St. Jude Children’s Research Hospital

Jennifer G. Alessi, MA, CHCP, Senior CME Manager, St. Jude Children’s Research Hospital

Angelina Kuo, JD, Director, Clinical Education and Training, St. Jude Children’s Research Hospital

Betsy W. Williams, PhD, MPH, FSACME, Clinical Program Director, Professional Renewal Center

Background/context/inquiry question
The St. Jude Children’s Research Hospital (SJCRH) Clinical Faculty Development (CFD) workshop series was created to address gaps in our clinical faculty’s skills in working with, teaching, and mentoring clinical trainees. Ongoing needs were identified via an ACGME site survey and faculty self-survey within the core competency areas of professionalism and individual burnout/well-being. With this initiative, we are exploring the effect of a longitudinal educational intervention that uses a variety of educational methods. Outcomes data will include pre/post physician self-rating of wellbeing and burnout and others’ judgment of aspects of professional comportment.

Reference to current literature/perspective on the topic
In the post-pandemic clinical setting, many clinicians report burnout and moral distress, both of which can impact their ability to behave in a professional manner when interacting with other clinical providers and patients. A lack of professionalism has been associated with numerous negative sequela including patient safety concerns, decreased morale of healthcare team members, and poor role modeling for trainees (Acad Med 2012;87[7]:845–852).

Theoretical framework(s)
Principles of informed self-assessment, self-directed learning, and the PRECEDE model of behavior change will all direct the structure of this educational intervention. The initial self-assessment will also gauge the participants’ perception of self-efficacy (social learning theory) as well as to what degree they are ready to change (transtheoretical stages of change model).

Methods
The educational intervention is structured as a Performance Improvement CME (PI CME) activity with a subgroup of our clinical faculty. Stage A: Initial data are derived from a recent ACGME site study that resulted in a citation in the area of professionalism. Year-end evaluation results from the SJCRH CFD series indicated the need for further education and concrete strategies on addressing burnout and creating healthy relationships in the clinical setting. Participants will participate in self-assessment using a validated instrument asking them to reflect on their health/wellbeing and their views on their skills in aspects of professionalism and interpersonal and communication skills. Stage B: The intervention will be of a longitudinal design utilizing a variety of educational formats including didactic sessions, small group discussion and experiential elements with modeling and role plays with feedback. Stage C: Participants will re-assess themselves, compare their results to the initial assessment, and reflect on results and then set ongoing professional development goals for themselves moving forward. Participants will move through the intervention as a cohort, providing the opportunity to develop a community of practice and support each other’s improvement.

Impact/relevance to the advancement of the field of CME/CPD
Although much research has been done in the area of professionalism and professional development in CME/CPD, the PI CME model allows us to approach this intervention in a unique way. As we move forward, we will explore ways in which to assess and measure our professional development program and its effect on clinical practice.

Preliminary Findings
Results from SJCRH CFD evaluations show an ongoing need for educational interventions in this area. At the time of the conference we will have self-reported data and will also be able to provide preliminary data from Stage B.

15. Assessing Workplace Bias Towards Internationally Trained Health Professionals and its Impact on Resilience and Wellness: A Needs Assessment Study 

Author(s)

Petal Abdool, MD, FRCPC, Medical Director Simulation Centre, Assistant Program Director, International Medical Graduates, Centre for Addiction and Mental Health

Rachel Antinucci, MHE, Research Analyst, Centre for Addiction and Mental Health

Rola Moghabghab, RN (EC) PhD GNC(C), Director Practice Innovation Standards and Measurement, Centre for Addiction and Mental Health

Umberin Najeeb, MD, FCPS (Pak), FRCPC, Associate Professor of Medicine, Faculty Lead Equity & Co-Director Master Teacher Program, University of Toronto

Sophie Soklaridis, PhD, Scientific Director and Senior Scientist, Centre for Addiction and Mental Health

Sanjeev Sockalingam, MD, MHPE, FRCP(C), FACLP, Chief Medical Officer, Vice-President Education and Senior Scientist, Professor, Department of Psychiatry, Centre for Addiction and Mental Health and University of Toronto Department of Psychiatry

Work in Progress

Background/context/inquiry question
Health Force Ontario (2020) shows a changing landscape with more than 10% of our workforce consisting of Internationally Educated Health Professionals (IEHP). Due to pandemic induced nursing shortages, this number has grown recently with the onboarding of nurses trained abroad becoming a necessity (Health Force Ontario, 2020). As more IEHPs join the Canadian healthcare workforce, the potential for experiencing discrimination increases. The goal of this project is to conduct a needs assessment of IEHP nurses and physicians to gather information on experiences of discrimination and identify the need for future educational training to address these issues.

Reference to current literature/perspective on the topic
International Medical Graduates (IMG) and Internationally Educated Nurses (IEN) experience discrimination based on their social identities including their gender, ethnicity, training location and professional status (Neiterman, 2015). Both IMGs and IENs report experiencing discrimination from their own professional group, patients, families or team members (Neiterman, 2015). These forms of bias and discrimination can make IEHPs transition to practice in Canada a very challenging and overwhelming experience. In a qualitative needs assessment study by Najeeb et al., (2019), IMG residents shared feelings of being ‘othered’ by peers and educators and also at a systemic level. They found a need for faculty development training initiatives for educators to incorporate ideas such as intersectionality and cultural safety.

Theoretical framework(s)
This project uses structural-level theories of discrimination to understand the contemporary patterns of racism and anti-immigrant sentiment in a Canadian context. We know that due to the focus on security and global terrorism following September 11, 2001, large-scale migration and a revival of nationalism have created a situation that systematically works against migrants (Castles, 2014).

Methods
We have conducted semi-structured one-on-one interviews with IEHPs within CAMH. We used a combined theoretical and opportunistic sampling strategy, sending information to all IEHPs via email from the Research Analyst. Interviews were completed with 8 nurses and 5 physicians over WebEx, and were 30-45 minutes in duration. All interviews were audio-recorded and transcribed. Thematic analysis of interviews using the Braun and Clarke (2006) approach is currently underway.

Impact/relevance to the advancement of the field of CME/CPD
The aim of this project is to identify the need for continuing professional development (CPD) training to address these issues. There is an urgent need to create CPD curricula to address these biases, discrimination and racism in order to create culturally safe work environments for an increasing number of IEHPs in healthcare settings.

Preliminary Findings
Thematic analysis of transcribed data is currently in progress. A prequel study was completed with 20 IMG residents concluding that they were in need of wellness supports, networking, mentorship, and increased support around discrimination and micro-aggressions. Early findings from this study emphasize the different experiences of transitioning and working in the Canadian healthcare setting between nurses compared to physicians. It is possible that hierarchical differences in occupational status are contributing to the diverse experiences of discrimination faced by nurses, residents and physicians. However, both nurses and physicians have highlighted the need for future change in the process of transitioning to a Canadian healthcare setting and the opportunity for educational advancements addressing discrimination in the workplace.

16. How Health Professionals Must Adapt their Clinical Practice in a Climate Crisis: A Needs Assessment Study

Author(s)

Haifa Akremi, PhD, Postdoctoral fellow, Continuing Professional Development, Faculty of Medicine, University of Montreal

Vincent Jobin, MD, Director, Continuing Professional Development, Faculty of Medicine, University of Montreal

Claudel P-Desrosiers, MD, Assistant Clinical Professor, Faculty of Medicine, University of Montreal

Eric Notebeart, MD, Adjunct Professor, Faculty of Medicine, University of Montreal

Nicolas Fernandez, MD, Adjunct Professor, Faculty of Medicine, University of Montreal

Work in progress

Background/context/inquiry question
Around the world, the climate crisis is generating significant increases in morbidity as well as heat stroke and cardio-respiratory diseases secondary to air pollution due to excessive greenhouse gas emissions and frequent heatwaves. There is a growing call for Health Care Professionals (HCPs) to adapt their clinical practices to attenuate the adverse effects of the crisis.

Reference to current literature/perspective on the topic
Increasing numbers of HCPs and managers acknowledge the urgency to act proactively. They need to adopt sustainable practices that will lead them to reduce their ecological imprint. HCPs will be increasingly required to safeguard patients from the harmful effects of climate change as well as contribute to building climate resilient health systems.

Theoretical framework
In order to effect change in organizational settings, we selected Giddens’ social structuration theory (1984), in which social outcomes are a function of the interplay between social structure and human agency. Structuration theory posits interaction among three dynamic variables: interpersonal relationships, professional identities, and the culture and rules that govern behavior. A better understanding of these variables will enable us to identify factors that can be used to facilitate adoption of new practices by HCPs.

Methods
The present study is piloted by an interdisciplinary team composed of members from the direction of Continuing Professional Development at the University of Montreal, experts on health impacts of climate crisis and professional representatives of allied health professions. The team will collect data from professionals in practice to identify the issues and challenges for each profession. This will allow the team to envision ways to empower professionals given their organizational culture and constraints, professional identities and interpersonal relationships, including with patients and families.

Impact/relevance to the advancement of the field of CME/CPD
The results of our study will allow our educational experts to design new pedagogical content and devise more effective dissemination strategies and tools. These will be offered through a dedicated CPD platform dedicated to HCPs. This educational content and the dissemination platform may contribute to sustainability and resilience of the Healthcare System in Quebec.

Preliminary Findings
The research project is actually in progress. Our team will be able to present the protocol and preliminary results about HCPs needs in person in March 2024.

17. Developing a Template for Longitudinal System-Integrated CME Applications Using Traditional and Novel Tools to Improve Care Across Multiple Settings—An In-Progress Report

Author(s)

Michael Fordis, MD, Sr Associate Dean CPD, Director of Ctr for Collab & Interactive Tech,

Director, Institute for CPD – QIPS, Baylor College of Medicine

Tresa McNeal, MD, Executive Vice Chair, Department of Medicine, Baylor Scott & White Central Texas

Derek Meeks, MD, MS, Informaticist, Associate Professor of Internal Medicine, Baylor College of Medicine

Jacob Minor, MD MS FACS, Physician Analytics specialist, Baylor Scott & White Health

Daniel Murphy, MD, MBA, Associate Professor, Department of Internal Medicine, Baylor College of Medicine

Michael McNeal, MD, MSc, Division Director, General Internal Medicine Department of Medicine, Baylor Scott & White Central Texas

Jason King, PhD, Associate Director, Educational Evaluation, Center for Collaborative and Interactive Technologies, Baylor College of Medicine

Michael O’Connor, BA, Assistant Director, Division of Digital Health Innovation, Ctr Collab & Interactive Tech, Baylor College of Medicine

Abraham Mendoza, MBI, Project Manager, Div Digital Health Innovation, Ctr Collab & Interactive Tech, Baylor College of Medicine

Sara Bedrose, MD, MSc, Assistant Professor, Department of Medicine, Baylor College of Medicine

Medha Airy, MBBS MPH, Assistant Professor, Department of Medicine, Baylor College of Medicine

Arindam Sarkar, MD, Assistant Professor, Department of Family and Community Medicine, Baylor College of Medicine

Jason Ramm, MD, Interim Chair, Department of Family Medicine, Baylor Scott & White Central Texas

Vamshi Punugoti, MS, Executive Director, Research and Data Technology, Baylor College of Medicine

Doug Compton, BBA, Senior, Manager,, Informatics, Research and Data, Baylor College of Medicine

Rory Laubscher, Senior Informatics Associate, Baylor College of Medicine

Kaylinh Ly, MPH, Sr Manager, Div of Continuing Professional Development, Ctr Collaborative an Interactive Tech, Baylor College of Medicine

Penny Coots, Director, Continuing Medical Education, Baylor Scott & White Health

Hania Janek, PhD, MSMEL, System Senior Vice President Clinical Medical Education, Baylor Scott & White

Ashok Balasubramanyam, MBBS, Vice President for Academic Integration, Baylor College of Medicine

Purpose/problem statement
Although much of the learning and needs for learning for practicing clinicians occurs in the workplace, traditionally continuing education often occurs elsewhere with limited impact on clinical outcomes. In response Price and colleagues proposed longitudinal models for CME, integrating education into practice in care systems, applying improvement interventions and implementation science, and focusing on outcomes, i.e., “System-Integrated CME (SysCME).” The purpose of this project is to explore the phased development and pilot testing of a SysCME template for outcomes-focused learning in the workplace in preparation for subsequent study of feasibility.

Approach(es)
We used conceptual models from learning theory, public health, quality improvement (QI), and implementation science to develop a SysCME template to guide the integration of CME across clinical workflows. For testing, we selected “resistant hypertension and screening for primary aldosteronism (PA),” a care gap aligned with our institutional priorities for blood pressure (BP) control, supported by literature, and confirmed clinically. PA is an often undiagnosed, treatable, and potentially curable cause of secondary hypertension. Our overall SysCME template includes sequential and longitudinal education aligned with the Precede/Proceed Model: ‘predisposing interventions,’ e.g., Grand Rounds and online modules in an indexed library suitable for just-in-time education; ‘enabling’ interventions using an internally developed point-of-care (POC) CME approach delivered via the EHR in clinical context using a best practice advisory (BPA) identifying patients at risk for PA; and iterative ‘feedback’ for improvement and maintenance (in development) via QI projects (with MOC Part 4 credit). We used emails, grand rounds, and meetings to invite/remind primary care clinicians (in family medicine and medicine in a health science university [HSU] and in a large health system [HS]) about the project, educational resources, BPAs, and POC-CME. The specific aim was to increase screening rates for PA by 25% over baseline.

Findings
The two organizations took slightly different approaches in evaluating screening rates. The HSU examined patients meeting criteria for PA screening, and the HS examined PA screening across total patients seen monthly in primary care clinics. At baseline, clinicians appropriately screened 1.4% (HSU) and 0.015% (HS) of the selected patient populations and with the SysCME improved to 7.1% (HSU) (an over 5-fold increase) and 0.117% HS (an 8-fold increase). Applying z tests for independent proportions to each organization’s data indicated statistically significant improvements in screening behaviors across time: p < 0.001 for HSU and p < 0.001 for HS.

Discussion
Early results from developmental pilots using a template for a SysCME project show promise for longitudinally integrated CME supporting improvements in care. However, full outcomes, contributions of components, and longer-term maintenance remain to be assessed in the planned feasibility study.

Barriers/facilitations
Facilitators included alignment with organizational clinical priorities, strong institutional support by the clinical enterprise, and assembly of committed multidisciplinary collaborative teams across institutions. Barriers/opportunities included prioritization of informatic and analytic resources, turn-around, EHR drug classification schema, and BPA engagement data access.

Impact/relevance to the advancement of the field of CME/CPD
This pilot implementation project provides encouraging evidence for the impact of the longitudinal system-integrated CME model for improving clinician performance and warrants continuing investigation.

18. Bringing Humility, Curiosity and Listening Together with Equity, Diversity, Inclusivity, and Accessibility to Address Inequities in the Health System: A National Project

Author(s)

Jerry M. Maniate, MD, M.Ed, FRCPC, FACP, CCPE, CPC(HC), Associate Professor of Medicine, University of Ottawa, Founding Director, Equity in Health Systems Lab

Cristian Rangel, PhD, Assistant Professor, University of Ottawa

Cassandra Barber, MA, PhD(C), Readiness Assessment Lead, Equity in Health Systems Lab

Heather MacNeill, MD, BSc(PT), MScCH (HPTE), FRCPC, Associate Professor, University of Toronto

Tamara Carver, PhD, Associate Professor, McGill University

Arun Radhakrishnan, MD CM, CCFP, MSc, Assistant Professor, Department of Family Medicine, University of Ottawa

Kannin Osei-Tutu, MD, MSc, CCFP, Clinical Assistant Professor & Senior Associate Dean for Health Equity and Systems Transformation, University of Calgary

Aimée Bouka, MD, M.Sc, DT&M, CCFP, Lecturer, Equity in health Systems Lab

Wendy Chong, MHA, Manager, Equity in Health Systems Lab

Bukola Salami, RN, MN, PhD, FCAN, Adjunct Professor, University of Calgary

Constance H. LeBlanc, MD, FCFP(EM), MAEd, CCPE, MBA, Professor, Department of Emergency Medicine, Dalhousie University

Marilee Nowgesic, RN, Executive Director, Canadian Indigenous Nurses Association

Problem statement
Our health system is being strained by several concurrent stressors and compounded by the numerous examples where racism, discrimination, and oppression have created spaces that are unsafe, lack compassion, and lead to poorer health outcomes for many people who live in Canada.

Methods/approach(es)
The Team Primary Care (TPC) Project aims to accelerate transformative change in the way primary care practitioners train to work together in Canada. The Equity, Diversity, Inclusivity, and Accessibility (EDIA) Cross-Cutting Theme supports primary care teams to understand their contexts through a unique approach, and to address the inequities and injustices that impact all people within the health system. The EDIA Theme elements were designed utilizing principles of education, quality improvement, and mentorship to support cultural transformation and will be both adapted into primary care professional training programs and also used to support practicing primary care teams.

Findings
A different approach to support cultural transformation that incorporated best practices for change management and our unique value-based approach was designed. The EDIA Theme Model starts with an EDIA Readiness Assessment Tool that then guides into the digital learning platform that links to people, interactive resources, and tools to support addressing educational needs. Individuals and teams identify a project and have the opportunity to be supported through voluntary participation with our national adaptive mentorship network with the aim of creating robust communities of practice. The initial engaging teams will then serve as support for others aspiring to begin this critical journey.

Discussion
The simultaneous development of several interconnected tools and resources required the creation of interconnected working groups comprised of subject matter experts (including those with lived experience), methodology experts, educational and instructional design experts; advisory committees that included stakeholders representing diverse communities in the health system including patient partners; the utilization of a scholarly developmental evaluation process; and the intentional attention to ensuring psychological safety to support diverse perspectives in sharing their voices in the trust-building process.

Barriers/facilitations
To address the historical silos in the health system required intentionally creating safe and supportive linkages between stakeholders to support communications, relationship-building, and establishing and then nurturing trust. The national grant provided the necessary finances to stimulate a broad system-level approach to support collaborative co-design. The trust-building has created the opportunity for bringing organizations together to partner and implement strategies to address the inequities that have long affected the health system. The resources of the EDIA Project will be sustained and built upon through the stewardship and guidance of the Equity in Health Systems Lab.

Impact/relevance to the advancement of the field of CME/CPD
This project underscores key learnings that should be considered for the development and creation of CPD that supports complex and sensitive processes such as EDIA. This includes creating a space for open dialogue; creating regular opportunities for diverse stakeholders to actively engage in the steps of the project; including patient partners and subject matter experts into the developmental process; underpinning the process with a unique approach; and including a willingness to make iterative improvements to the tools and resources as feedback is gathered.

19. Likelihood Continuing Medical Education Reduces Disciplinary Recidivism Among Physicians

Author(s)

Elizabeth F. Wenghofer, PhD, Full Professor, Faculty of Education and Health, Laurentian University

Katie Arnhart, PhD, Research Project Manager – Research and Data Integration, Federation of State Medical Boards

Xiaomei Pei, PhD, Research Project Manager – Research and Data Integration, Federation of State Medical Boards

Aaron Young, PhD, Vice President, Research & Data Integration, Federation of State Medical Boards

Background/purpose/inquiry question
State medical boards are charged through their medical practice acts to regulate physician practice and, when necessary, discipline physicians for incompetent or inappropriate behavior. Boards often authorize continuing medical education (CME) as part of a disciplinary action; however, it is unclear how effective remedial CME is in reducing the likelihood of physicians receiving subsequent discipline. Our study examined the relationship between remedial CME required by state medical boards and subsequent disciplinary actions.

Theoretical framework
The study builds on the Cambridge Model of physician performance and competence, which recognizes that physician performance is complex, multidimensional and influenced by various factors that extend beyond CME alone.

Methods
The national-level sample included 4,061 MD-physicians whose initial disciplinary event included license restrictions, probation or other conditions imposed by state medical boards between 2011 and 2015. A multivariate logistic regression model examined whether physicians required to complete CME as part of their discipline process were less likely to receive another disciplinary action within five years (dependent variable). Our main independent variable was the requirement to complete CME (yes/no) as part of a physician’s first disciplinary action in addition to their license restrictions, probation, or other conditions imposed. Covariates in the study included controlling for demographic and educational variables: sex, physician age, international medical graduate status and ABMS certification. The reason(s) for first discipline was extracted from official board order documentation from state medical boards.

Results/findings
Of the 4,061 physicians, 36% (n = 1,449) were required to complete CME as part of their initial discipline process, and 35% (n = 1,426) received another disciplinary action within five years. After accounting for other factors, physicians who were required to complete CME as part of their discipline were less likely to receive another disciplinary action (OR = 0.597, 95% CL = 0.513-0.696, p < 0.001) within five years compared to disciplined physicians who were not required to complete CME.

Discussion
Our study supports that CME can be a helpful remedial action for certain physicians to reduce the risk of disciplinary recidivism. Physicians required to complete CME as part of their initial discipline process were less likely to receive additional disciplinary action by state medical boards within five years.

Limitations
While our analysis broadly supports that CME reduces the risk of physicians receiving additional action within five years, there is a caution that CME may not be an appropriate option to mitigate risk for various situations. Although we cannot claim a causal effect between CME and a reduced probability of recidivism, our results are consistent with trends in the literature showing the potential benefits of remedial CME for improving physician practice.

Impact/relevance to the advancement of the field of CME/CPD
Our findings help contribute to disciplinary and CME research by finding that among physicians with certain disciplinary actions, requiring CME as part of their discipline can reduce their likelihood of additional discipline. Our research also helps shed light on the complex issues between physician discipline, remedial CME and recidivism and how future studies using big data may inform regulators on how to better keep physicians practicing well and safely.

20. Presences and Absences: Exploring Social Processes and Paradigms of Education in Continuing Interprofessional and Quality Improvement Education

Author(s)

Joanne Goldman, PhD, Education Scientist, University of Toronto

Farah Friesen, MI, Manager, Research & Knowledge Mobilization, Centre for Advancing Collaborative Healthcare & Education, University of Toronto

Tara Burra, MA, MD, FRCPC, Physician, Education Lead, Sinai Health and Centre for Quality Improvement and Patient Safety, Toronto, Ca

Lindsay Baker, MEd, Associate Director, Curriculum Integration & Partnerships, Centre for Faculty Development, St. Michael’s Hospital, Toronto, Ca

Maria Tassone, MSc, BScPT, Educational Director, Education & Professional Development, University Health Network, Toronto, Ca

Stella Ng, PhD, Education Scientist, Centre for Advancing Collaborative Healthcare and Education, University Health Network, Toronto, Ca

Background/purpose/inquiry question
Social processes and structures (e.g., power relations, gendered roles, institutional racism, organizational norms) influence interprofessional collaboration (IPC) and quality improvement (QI) practices. Yet, continuing professional development (CPD) for IPC and QI could pay more attention to these processes and structures when designing curricula and pedagogy . This study aimed to increase understanding of the content about social processes and structures relevant across continuing interprofessional and QI education, and the paradigms of education informing how this content is taught.

Theoretical framework(s)
This study uses the conceptual framework of paradigms of education. Paradigms of education refer to the philosophical underpinnings that influence the assumed purposes of education prescribed roles and practices of teacher and learners, to serve targeted educational goals. In order to design and deliver the most effective continuing professional development (CPD) offerings, educators must carefully consider which paradigms of education align with the social complexities that shape and constrain QI and interprofessional education and care.

Methods
We conducted a multiple instrumental case study methodology of two interprofessional education and one quality improvement CPD courses across two academic health science centres. We conducted 28 hours of observations; 13 interviews with Centre leaders, course directors, and course teachers; and collected program materials (e.g., slides, images or videos used in class). We conducted a thematic analysis of the data using paradigms of education as sensitizing theoretical concepts.

Results/findings
Content about social processes and structures appeared in courses across both Centres. Such content included, for instance, a session on health equity and a discussion about power in patient engagement in the QI course and content about interprofessional power dynamics and hierarchies and sexism in interprofessional interactions in the IPE courses. The particular foci varied, though, reflective of the literature in the interprofessional and QI fields and the ways that concepts such as equity are currently being examined. In relation to paradigms and practices of education, the focus was largely on delivery (e.g., cases, small group breakouts) rather than on pedagogies, although there was increasing focus on how to incorporate a transformative paradigm when teaching content on social processes and structures. Faculty expressed struggles balancing content on foundational tools and frameworks with critical knowledge.

Discussion
There is commitment amongst QI and interprofessional CPD course leaders to attend to content and pedagogy related to social processes and structures; however, the faculty identified ongoing challenges in making changes and incorporating paradigms of education. Our study provides guidance on addressing absences identified related to social processes and structures, for both content and educational paradigms enacted.

Limitations
Data collection was limited to two centers in Canada and we focused on data from course/centre leaders (i.e. not learners).

Impact/relevance to the advancement of the field of CME/CPD
There is opportunity to further mobilize scholarship from critical social sciences and education science to extend how pedagogical approaches, particularly transformative paradigm, are used in interprofessional and QI-CPD.

21. Interprofessional Continuing Education as a Strategy to Increase Clinician Readiness to Use Artificial Intelligence in Their Practice

Author(s)

Marianna Shershneva, MD, PhD, CPD Evaluation and Assessment Specialist, University of Wisconsin School of Medicine and Public Health

Barbara Anderson, MS, Director, Office of Continuing Professional Development; Chair, Interprofessional Continuing Education Partnership, University of Wisconsin School of Medicine and Public Health

Background/context/inquiry question
Artificial intelligence (AI) is a fast-growing tool in the digital transformation of healthcare. AI can decrease time spent on administrative tasks, streamline workflows, advance the diagnosis, treatment, and monitoring of patients, and reduce costs. There are challenges in the development of AI algorithms and the evolving use of AI tools in patient care. Our institution is piloting generative AI tools, such as patient messaging. This study explores how continuing education can prepare healthcare teams for the use of AI in their practice.

Reference to current literature/perspective on the topic
Researchers agree that most clinicians are insufficiently prepared for the adoption of the latest developments in AI. Recent publications call for a new infrastructure for learning about AI and the engagement of educators outside of the medical community. A qualitative study using expert interviews generated a comprehensive list of AI-related competencies [Russell RG, et al. Acad Med. 2023;98(3):348-356.]. We used the latter as a springboard to design a competency-based needs assessment to inform our project.

Possible theoretical framework(s)
A six-step approach guides the development of a curriculum to support the institution’s efforts to utilize generative AI in the clinical setting [Kern DE, et al. Curriculum development for medical education: a six‑step approach. The John’s Hopkins University Press; 2009.]. We are also exploring AI theories when selecting learning formats, content, and practice change measures [Kaliraj P, Devi T (Eds.). Artificial Intelligence Theory, Models, and Applications. Taylor & Francis Group; 2022.]

Possible methods
Needs assessment included a written survey of last year’s participants in our continuing education program and key informant interviews. We have assembled a planning committee to develop a multifaceted and longitudinal educational intervention. Assessments and evaluation will include research tools to document the educational impact and answer the research question, “What interprofessional education strategies effectively teach AI concepts and their application to patient care?”

Preliminary Findings
Survey respondents were asked to rate their current and desired AI abilities using a scale from 1=low to 5=high. Responses from 502 clinicians, 30% of whom were physicians, are reported as “current/desired” means for each statement. Explain what AI is: 2.7/3.8. Identify the range of health-related AI applications: 2.0/3.8. Appraise the ethical issues for clinicians, patients, and populations raised by various design, implementation, and use scenarios involving AI: 2.1/3.9. Explain to patients the concepts of risk and uncertainty as they relate to the outputs of AI-based tools and describe practical implications for their care: 2.0/3.9. Access critical information about specific AI-based tools before applying them to patient care: 1.8/3.9. Identify potential biases in the design of an AI-based tool and the implications of those biases for patient care and population health: 1.9/3.9. Participate collaboratively in team-based discussions that analyze changing roles, responsibilities, and workflows associated with the adoption of novel AI-based tools: 1.9/3.7. Effectively use AI-based tools to facilitate critical communications between all members of healthcare teams: 1.8/3.8. Only 16% of respondents participated in AI-focused CPD and/or practice-based improvement activities.

Impact/relevance to the advancement of the field of CME/CPD
This study will help define the role of CPD in the clinical adoption of AI and may suggest effective strategies for future education.

22. Confidence in the Practice of Secure Recovery Through Education of Forensic Staff

Author(s)

Shaheen A. Darani, MD, FRCPC, Director, Faculty Development & Asst Professor, Dept of Psychiatry, Associate Director, PG Wellness, Temerty Faculty of Medicine, University of Toronto

Sandy Simpson, MBBS, FRCPC, Chair, Forensic Psychiatry, Department of Psychiatry, University of Toronto

Stephanie Penney, PhD, Independent Scientist & Associate Professor, Centre for Addiction and Mental Health, Department of Psychiatry, University of Toronto

Courtney Brennan, MSc, OT, Occupational Therapist, Centre for Addiction and Mental Health

Remar Mangaoil, RN, PhD, Professor, Cambrian College

Faisal Islam, PhD, Manager, Education, Evaluation and Quality Improvement, Center for Addiction and Mental Health (CAMH)

Treena Wilkie, MD, FRCPC, Chief of Forensic Psychiatry & Associate Professor, Centre for Addiction and Mental Health, Department of Psychiatry, University of Toronto

Background/context/inquiry question
Recovery is the process of personal change leading to a satisfying, hopeful, and contributing life, even within the limits of mental illness. Recovery-oriented care has become a dominant paradigm in mental health service provision and over the past decade has increasingly been applied to the forensic setting (i.e., “secure recovery”). While recovery-oriented models are well developed, these have not yet been fully adapted to the forensic recovery setting nor yet formally taught to care providers. Our project aims to identify knowledge, skills, and education needs of forensic psychiatric staff in relation to the practice of secure recovery. The overarching goal of this project is to develop a tailored curriculum in secure recovery that is informed by CPD best practices and the learning needs of forensic staff. As our project aims to build capacity and confidence in in recovery-oriented practice through CPD education. It is anticipated that this will support the practice of compassionate patient care amongst interprofessional forensic staff. Recovery-oriented practice is an example of compassionate care with the common goal of ensuring people with mental illness retain a sense of hope and optimism so that they can live with and gain control over their debilitating symptoms associated with enduring illness.

Reference to current literature/perspective on the topic
There is limited evidence on the practice of secure recovery, and in particular, about forensic care providers’ knowledge, skills, and education needs in relation to this practice. There is insufficient knowledge on the practical challenges around implementing recovery-oriented care in secure settings with consistency and fidelity, and specifically, developing the therapeutic alliance effectively, with this particular patient group. Kennedy (2022) calls for the development of a secure recovery curriculum to fill these gaps in knowledge.

Theoretical framework(s)
The theoretical frameworks used in the design of the needs assessment and future education included the principles of adult learning and experiential learning.

Method(s)
A needs assessment survey has been administered to all forensic psychiatrists, forensic psychiatry fellows, general psychiatry residents, nursing, and allied health staff in the forensic division at CAMH (n=300). Surveys include both closed and open-ended questions and administered electronically through a REDCap (Research Electronic Data Capture). The survey aims to identify the knowledge, skills, and education needs among forensic psychiatric care staff in relation to the practice of secure recovery. Descriptive statistical techniques (e.g. frequencies, crosstabs and chi-squares where applicable) were used to analyze the survey data and thematic analyses will be conducted to analyze the open ended question data. Data from our needs assessment will help inform the development of a CPD curriculum in secure recovery.

Preliminary Findings

Of 108 responses, 45% were nursing staff. Staff forensic experience ranged from 0 to 43 years with a median of 5 years. Results showed 79% reported good or excellent knowledge in recovery-oriented principles, however, 44% were not, somewhat, or only moderately confident in their skills in implementing recovery-oriented care; 59% did not believe they received adequate education and 93% were interested in secure recovery education.

Impact/relevance to the advancement of the field of CME/CPD
Results will inform the development of future secure recovery continuing CPD curriculum to boost the confidence of forensic interprofessional staff.

23. Integrating NLP into Equity, Diversity, Inclusion, and Indigeneity Frameworks

Author(s)

Kayle Donner, MA, M.Ed., Research Methods Specialist, Center for Addiction and Mental Health (CAMH)

Faisal Islam, PhD, Manager, Education, Evaluation and Quality Improvement, CAMH

Marta Maslej, PhD, Staff Scientist, Krembil Centre for Neuroinformatics, CAMH

Kenya A. Costa-Dookhan, MSc, Medical Student, University of Toronto Temerty Faculty of Medicine

Anupam Thakur, MBBS, MD, MSc, Staff Psychiatrist, University of Toronto Department of Psychiatry, Education Scholar, CAMH

Heba Baig, BSc, Research Analyst, City of Toronto

Sanjeev Sockalingam, MD, MHPE, FRCP(C), FACLP, Chief Medical Officer, Vice-President Education, and Senior Scientist, Professor, Department of Psychiatry, Centre for Addiction and Mental Health, University of Toronto Department of Psychiatry

Work in Progress

Background/context/inquiry question
Natural Language Processing (NLP) offers a promising set of techniques for efficiently analyzing large volumes of unstructured text data. This project proposes a novel method for incorporating NLP analyses into program evaluations conducted under EDII frameworks. The method is especially suitable for analyzing learner feedback from qualitative survey data collected after CME/CPD.

Reference to current literature/perspective on the topic
The use of NLP in evaluating CME/CPD programs is gaining in popularity, but little guidance is available for investigators seeking to integrate NLP techniques with guidelines for analyzing data through an Equity, Diversity, Inclusion, and Indigeneity (EDII) lens.

Theoretical framework(s)
This project aims to integrate NLP analyses into the Health Equity and Inclusion Framework for Education and Training (Agic et al., 2021), as well as the Toolkit for Centering Racial Equity Throughout Data Integration developed by the Actionable Intelligence for Social Policy program at the University of Pennsylvania.

Methods
Qualitative feedback from 312 learners in a CPD training on anti-Black racism was analyzed using NLP. Learners were asked about their intention to change their practice in response to the training, as well as suggestions for improvement. Responses were segmented into single sentences, which were then grouped into clusters using the BERTopic algorithm. The resulting clusters were manually analyzed using Braun and Clarke’s (2006) guidelines for thematic analysis, and those deemed especially large and heterogeneous in meaning were re-clustered and re-analyzed. Responses in each cluster were then disaggregated based on intersecting demographic variables before final thematic analysis.

Impact/relevance to the advancement of the field of CME/CPD
As AI technologies are increasingly adopted in health care, there is a need to ensure these technologies are used in ways that promote health equity. The iterative approach proposed here relies on human judgment at multiple stages throughout the process, allowing investigators to ensure their use of AI is consistent with their broader goals and values.

Preliminary Findings
Evaluation surveys distributed to CPD learners often yield very short responses, in numbers great enough to make manual qualitative analysis unfeasible. When clustering single sentences algorithmically, responses tend to be grouped based on the presence of a small number of key words or phrases. For example, responses containing “race”, “racism”, or “anti-Black” were grouped together. Re-clustering these responses yielded more specific and interpretable topics, including requests for more practical strategies for addressing racism, as well as commitments to take action when racism is encountered. Disaggregating the algorithm’s results using demographic variables has also shown some group differences in the use and context of certain key words and phrases.

24. Building an Equity Lens in AI Education: Results from a Scoping Review

Author(s)

David Wiljer, BA, PhD, Executive Director – Digital Education and IT, University Health Network, Toronto, Ca

Melody Zhang, MSc, Research Analyst, University Health Network, Toronto, Ca

Inaara Karsan, MHI, Senior Project Analyst, University Health Network, Toronto, Ca

Shadia Jirreh, HBSc, Research Assistant, University Health Network, Toronto, Ca

Rebecca Charow, MPH, PhD(c), Research Associate, University Health Network, Toronto, Ca

Bemnet Teferi, MPH, Research Analyst, University Health Network, Toronto, Ca

Tharshini Jeyakumar, MHI, Research Associate, University Health Network, Toronto, Ca

Maram Omar, MSc, Research Analyst, University Health Network, Toronto, Ca

Bryn Davis, Research Assistant, University Health Network, Toronto, Ca

Divya Kamath, Practicum Student, University Health Network, Toronto, Ca

Background/purpose/inquiry question
Healthcare is in the midst of an artificial intelligence (AI) revolution. However, a gap exists in understanding how to mobilize principles of equity throughout the lifecycle of AI integration. Education and continuing professional development (CPD) are critical to facilitating AI-enabled healthcare that is accessible, inclusive and equitable. Yet, previous reviews identified a gap in AI education at the CPD level. Moreover, existing AI competency frameworks have not been designed with how AI could be used to enhance health equity. This work aimed to explore and synthesize the AI medical education literature to identify competencies needed to advance equity in AI implementation.

Theoretical framework(s)
The National Academy of Medicine’s (NAM) Quintuple Aim of Equity and Inclusion outlined equity and inclusion as clear goals during AI development and deployment phase. Informed by NAM’s goals, this study aims to identify competencies and recommendations from the literature to provide educational guidelines about equitable AI use.  Additionally, the Socioecological Model was used to understand the implications for promoting equitable access to health care at the micro-, meso-, and macro-level.

Methods
This scoping review followed the Arksey and O’Malley methodological framework. A grey literature search was also conducted to identify non-academic sources (e.g., government reports). Only articles and grey literature published between 2012 and 2022 were considered for review to account for increased interest in adoption of AI technology in healthcare. Articles were included if they: (1) are in English, (2) discuss competencies that advance health equity principles in AI implementation, (3) describe AI implementation training programs for clinicians, and (5) discuss best practices on inclusion, diversity, equity, and accessibility for AI implementation.  Two research analysts conducted the two-stage screening process (title/abstract scan and full-text review), data extraction and analysis. Extracted data included general article details, education/training programs, perspectives towards education, skills and competencies, facilitators/barriers to education program implementation and health equity principles into education programs. Extracted data underwent deductive and inductive thematic analysis guided by the Socioecological Model.

Results/findings
From the 4,130 peer-reviewed articles screened, 11 articles met the inclusion criteria and were analyzed. Additionally, from the 50 grey literature sources identified, 22 met the criteria for data extraction and analysis. The scoping review identified major topics of discussion in ethical AI implementation, including: (1) five important principles of equity in AI adoption and implementations were identified from health organizations; (2) call to action for AI education to advance equity principles and transparency among clinicians; (3) highlighted education and training strategies to support equity principles; and (4) challenges integrating equity in AI education and implementation.

Discussion
Findings highlight the current gap in equity principles within AI education in health. This study presents practical implications to advance health equity by promoting ethical AI use and responsible implementation standards. These implications can be leveraged in the development of current and future AI curricula for HCPs.

Limitations
n/a

Impact/relevance to the advancement of the field of CME/CPD
While the field of AI in healthcare is evolving rapidly, issues of biases and inequities in AI persist. CPD interventions are required to build the capacity of clinicians to adopt these changes and use these tools.

25. Developing Leaders to Accelerate the Appropriate Adoption of Artificial Intelligence in Health Care

Author(s)

Bemnet Teferi, MPH, Research Analyst, University Health Network

Tharshini Jeyakumar, MHI, Research Associate, University Health Network 

Rebecca Charow, MPH, PhD(c), Research Associate, University Health Network

 Azra Dhalla, BA, MBA, Director Health AI Implementations, Vector Institute

Jessica Jardine, BSc, B.Ed., Project Manager, University Health Network

 Sedef Kocak, B.Eng., MBA, MSc., PhD, Director of Professional Development, Vector Institute

 Jane Mattson, Manager of Continuing Education, Michener Institute

 Maram Omar, MSc, Research Analyst, University Health Network

 Mohammad Salhia, HBSc, M.Ed., Managing Director, Rotman School of Management

Jillian Scandiffio, BSc., MSc, Research Coordinator, St. Michael’s Hospital

David Wiljer, BA, PhD, Executive Director – Digital Education and IT, University Health Network

Background/context/purpose
Healthcare leaders play a pivotal role in the integration of artificial intelligence (AI) within health systems, yet AI literacy gaps persist. Tailoring education to their requirements empowers them to effectively integrate AI into healthcare strategies and drive transformative change. To address this, the AI for Healthcare Leaders certificate program aims to enhance understanding and management in AI adoption. This program evaluation identifies comprehensive and equitable education approaches, ensuring the seamless integration of AI-enabled care.

Theoretical framework(s)
Within the larger project scope, the design, evaluation and knowledge dissemination of the evidence-based AI education program drew from the Knowledge-to-Action (KTA) model. A qualitative study was integrated into this program evaluation, guided by the RE-AIM framework dimensions of Reach, Effectiveness, Adoption, Implementation and Maintenance and the Health Equity and Inclusion Frameworks to explore learning environments to support continuing professional development (CPD) and AI literacy.

Methods
The Leaders Program was conducted in three cohorts, taking place in January 2022, September 2022, and February 2023. Leading industry experts covering healthcare data analytics, AI engineering, cybersecurity, legal implications, and clinical oncology significantly contributed to the program’s content. The curriculum was structured to include dynamic interactive discussions, collaborative learning activities, and opportunities for reflection on real-world applications. Evaluation data collected included learner pre/post-surveys, post-interviews, and instructor/ tutorial assistant debriefs with the program developers. The interview data was analyzed following a deductive approach and triangulated with the debrief data. Descriptive statistics were derived from the survey data.

Results/findings
103 Canadian healthcare leaders attended the three cohorts. The program engaged various healthcare leadership roles as participants, spanning healthcare administrators, researchers, healthcare providers, patient partners, and other professionals such as business owners and consultants. A total of 18 (17.5%) participants completed interviews. Leaders’ perspectives on the program revealed five significant themes. (1) Knowledge mobilization and translation are required to foster an organizational culture that enables change. (2) Education focusing on critical reflection and supportive coaching is required for informed decision-making on AI implementation. (3) The understanding of AI through real-world healthcare use cases, enhancing AI preparedness, and practically applying course concepts. (4) Integration of patient perspectives into AI-driven healthcare initiatives emerged as a crucial aspect of the discourse.

Discussion
Healthcare leaders are essential actors in guiding the strategic course of AI implementation and utilization within healthcare. CPD has implications for fostering an organizational culture that’s not only receptive but prepared for AI implementation. These findings emphasize the need for a comprehensive understanding of AI’s impact on healthcare alongside technical skills. The evaluation insights stress the importance of robust AI education and its integration into CPD, thus enhancing AI implementation and patient care.

Impact/relevance to the advancement of the field of CME/CPD
CPD initiatives advancing the appropriate adoption of AI must focus on building the mindset, skillset and toolset of not only clinicians but health care leaders to effect appropriate change.

26. Impacts of Sex and Gender on Health and Disease: Where Do These Topics Fit in Continuing Medical Education?

Author

Kimberly Templeton, MD, Associate Dean, Vice Chair, University of Kansas Medical Center

Purpose/problem statement
Sex and gender impact all health conditions, from risks for disease, to presentation, to treatment outcomes. While much research does not disaggregate results based on sex or gender, available data that provides this information is infrequently included across the continuum of medical education. While recent studies have shown the lack of inclusion of sex and gender differences in undergraduate medical education, there has been no prior assessment of content in this area for continuing medical education. Familiarity by physicians of the impacts of sex and gender on health and disease are crucial in providing appropriate care for all patients.

Approach(es)
The survey of CME course directors, planning committees, and faculty were surveyed post hoc for the inclusion of sex-based differences. The impacts of gender were presumed by pre-existing questions regarding the social determinants of health and health care disparities. Discussion was held with CME staff regarding the importance of sex and gender and how to assess responses to these questions on the survey.

Findings
Less than half of course directors noted that consideration of sex was included in their continuing education courses. The primary explanations provided were that that was insufficient data to inform presentations and that that sex was not relevant to their topics. Survey responses from the larger group were found to indicate a greater likelihood of inclusion of this material in the content than the planning of the event.

Discussion
Similar to what has been found in UME, sex and gender concepts were not consistently included in CME activities.  While there needs to be enhanced attention placed on including information regarding the impacts of sex on health and disease across the continuum of education activities, to be consistent and sustainable, this requires a coordinated effort in planning, rather than relying on the interests of individuals planning committee members and/or speakers.  This includes recommending to speakers to provide comments when sex-based differences are not discussed within individual lectures because of lack of data. This will help participants understand the knowledge that is available, what areas requires additional research,  and that data from men alone may not always be extrapolated to women and gender minority persons.

Barriers/facilitations
Lack of awareness of the impact of sex and gender across all health conditions is impacting the inclusion of this in medical education across the continuum, often impacting the quality of patient care provided. Next steps include development of a recorded Q&A session with a sex and gender in medical education expert to approach topics such as the differences between sex and gender, examples of how these impact health and disease, and how to incorporate these into continuing medical education presentations. In addition, the session will discuss how to approach faculty to assure that this content is included and to point them to reliable resources for their presentations.

Impact/relevance to the advancement of the field of CME/CPD
For content from CME presentations to be relevant to all patients, data regarding sex and gender differences needs to be included. If this data is not available, faculty need to comment on that, so that participants understand that provided content may not be relevant to all patients.  In addition, participants should be aware that if they are extrapolating results of studies primarily including men to other populations or are using the results of studies in which there is no disaggregation of results based on sex or gender, patient outcomes may be challenging to predict.

27. Challenging the Status Quo: Quality Improvement to Overcome Disparities in Care for African Americans with Multiple Myeloma

Author(s)

Ronan O’Beirne, MBA, Director, Continuing Medical Education, Marnix E. Heersink School of Medicine, University of Alabama at Birmingham

Luciano Costa, MD, PHD, Associate Director for Clinical Research, O’Neal Comprehensive Cancer Center, The University of Alabama at Birmingham 

Katie Lucero, MS, PhD, Vice President, Audience, Analytics, Outcomes, Medscape Education

Haleh Kadkhoda, MS, Vice President, Strategic Partnerships and Public Health, Medscape Education 

Background/context/purpose
African Americans tend to have better prognosis with multiple myeloma (MM), but this scenario does not translate into better survival rates. System-level, HCP- and patient-related factors have an impact on patient outcomes. Unconscious and implicit biases as well as social determinants of health play a role in this observed disparity. Barriers to appropriate treatment and a lack of awareness of the disease characteristics among many HCPs and other care providers contribute to treatment approaches inconsistent with current recommended practice. University of Alabama at Birmingham (UAB) and Medscape Oncology collaborated to address the identified practice gaps by developing an education initiative for UAB’s Health System cancer associates who help manage clinical care for patients with MM. The goal was to improve clinical knowledge, increase awareness about disparities for African Americans with MM, and enhance care delivery for patients.

Theoretical framework(s)
Bronfenbrenner’s Ecological Systems Theory suggests individual behavior is complex because of the multiple levels of the environment from micro to macro. Therefore, we leveraged quality improvement (QI) methods to develop a program to foster more equitable clinical care for African American patients with MM.

Methods
This project employed mixed methods, combining both qualitative and quantitative methods to identify impact and continued needs. Nominal group technique identified care providers’ perspectives on barriers to patient care pre- and post-intervention. A baseline assessment of HCP knowledge, attitudes, and perceptions was conducted and repeated after the intervention. Results were then shared as part of UAB’s online training module about elevating equity in healthcare. A second online CME-accredited video expert discussion reviewed clinical presentation of racial disparities and access to healthcare.

Results/findings
At UAB, 52 hematologist/oncologist-focused HCPs and care providers participated in the educational training sessions. This QI project changed attitudes and beliefs, which are the root of many biases that may result in suboptimal practices in care for African American patients with MM. For example, participants were more likely to agree that African American patients are more likely to be symptomatic and have MM-related complications than White patients (53% pre to 84% post) and less likely to undergo stem transplantation (55% pre to 100% post). There was increased knowledge about cytogenetic alteration (50% pre to 75% post) and survival rates (45% pre to 65% post). Findings from nominal group sessions revealed providers recognized the value of external social support in ensuring good care and improved outcomes and recognized their need to understand patients’ individual circumstances.

Discussion
As an effort to promote best practices in facilitating empathy and compassion in healthcare, this quality improvement project pointed to a substantial knowledge gap among health care professionals about understanding and addressing MM in African Americans. Further insights into next steps to further close that gap were discovered.

Impact/relevance to the advancement of the field of CME/CPD
To promote empathy and compassion in care, it is necessary to overcome structural barriers, misconceptions and bias among healthcare professionals and organizations, and integrate elements of patient advocacy to improve access to timely, optimal and culturally appropriate care.

28. AI Education for Mental Health Professionals to Improve AI Adoption in Healthcare: A Program Evaluation

Author(s)

David Wiljer, BA, PhD, Executive Director – Digital Education and IT, University Health Network

Bryn Davis, Research Assistant, University Health Network

Bemnet Teferi, MPH, Research Analyst, University Health Network

Lydia Sequeira, PhD, Director, Service Programs and Innovation Implementation, Kids Help Phone

Kathleen Sheehan, MD, DPhil, FRCPC, Psychiatrist, Toronto General Hospital

Jane Mattson, Manager of Continuing Education, Michener Institute

Jessica Jardine, BSc, B.Ed., Project Manager, University Health Network

Rebecca Charow, MPH, PhD(c), Research Associate, University Health Network

Tharshini Jeyakumar, MHI, Research Associate, University Health Network

Maram Omar, MSc, Research Analyst, University Health Network

Divya Kamath, Practicum Student, University Health Network

Asha Maharaj, BA, MBA, Director, Community and Continuing Education, CAMH

Maureen Abbott, Manager, Mental Health Commission

Sheldon Mellis, BA, Senior Program Manager, Mental Health Commission

Background/context/purpose
The mental health and addiction care field has expressed hesitancy in regard to the adoption of artificial intelligence (AI). Yet there is a desire for an increase in educational opportunities, which must reflect and support the rapid change of AI. The educational program aimed to provide mental health professionals with foundational AI knowledge (machine learning, bias, etc.) and current applications of AI in mental health (case-based discussions). The larger study aimed to educate healthcare professionals (clinicians, leaders) on the adoption of AI in healthcare.

Theoretical framework(s)
The Knowledge-To-Action (KTA) framework was employed to shape the larger project that included this educational program. The program was guided by the Health Equity and Inclusion framework. The data analysis has focused on the impact of reflective learning in accelerating the adoption of appropriate AI into mental health practice.

Methods
The mental health program ran consecutively over four (4) weeks. Each two-hour class session consisted of didactic lectures, diverse guest speakers who presented their application of AI in mental health, pre-readings, discussion groups and a weekly reflection assignment. Certificates were awarded to participants who completed at least three (3) weekly reflection assignments. Data were collected through pre- and post-evaluation surveys, interviews and class artifacts (reflection assignments). The post-evaluation interviews employed purposeful sampling. A content analysis of the reflection assignments was conducted and complemented by an analysis of the depth of each reflection, using the analysis and meaning making portion of the REFLECT Rubric. Interviews were inductively coded, and triangulated with data from the surveys, and reflection assignments

Results/findings
83% (n=30) successfully received a certificate. 76.9% of participants were women and 50% identified as a racialized person. Among the participants their professions included nurses (42.3%), psychiatrists (11.5%), therapists (3.8%), healthcare administrators (19.2%) and others (23.1%). Key evaluation findings thus far include, (1) participants reported having an overall positive experience, yet they will need more education, in the form of example-based, hands-on training and organizational support before being prepared to implement AI into their workplace. (2) Learning from like-minded professionals assisted in greater knowledge transfer. By deploying mental health experts as guest lecturers and by tailoring the content to align with the participant’s careers, they felt able to engage in more meaningful conversation after completing the course; (3) The reflection assignments allowed participants to internalize the content, enhance their knowledge and curiosity, which led to participants continuing their professional development. Preliminary results of the reflection analysis are currently underway and will be presented at the conference.

Discussion
Mental health professionals expressed a positive reception of the specialty focused program. Moreover, it enhanced AI education for mental health professionals, with a focus on bridging the gap between current mental health practices and the potential for AI augmented practices through an emphasis on reflective, continual learning. Further AI education is necessary for clinicians to confidently implement AI into their practice.

Impact/relevance to the advancement of the field of CME/CPD
This program provides important insight at the intersection of mental health professionals continuing professional development, their conceptualization of AI and the impact of reflective learning.

29. A Novel Professional Development Training to Build Skills and Resilience in Facilitating a Code Lavender Debrief

Author(s)

Tucker Gordon, Simulation Educator, Center for Addiction and Mental Health

George James, Lived Experience Advisor, Center for Addiction and Mental Health

Rachel Antinucci, MHE, Research Analyst, Center for Addiction and Mental Health

Stephanie Sliekers, MEd, Manager, Simulation & Digital Innovation, Center Addiction and Mental Health

Shawn Lucas, Manager, Spiritual Services, Center for Addiction and Mental Health

Background/context/purpose
The Centre for Addiction and Mental Health (CAMH) has implemented Lavender Support to assist staff and physicians following a crisis event. Unlike immediate post-events, which are limited to staff involved, Lavender Support is available to all impacted staff, whether they’re directly or indirectly involved. This is delivered by CAMH’s Spiritual Services team, who has identified the need to develop and implement practices to facilitate Lavender Support debrief discussions and practices facilitation in a safe, supportive environment.

Theoretical framework(s)
Lavender Support, based on the Cleveland Clinic model, is an evidence-based intervention supporting staff and physicians to better understand a difficult situation, enabling problem solving to emerge as a result (Stone, 2018). Based on psychological first aid, it’s meant to address harm and assist people working through their experiences. At CAMH, Lavender Support focuses on the group process of healing and provides individual support and wellness programs.

Methods
With this unique model, CAMH’s Spiritual Services and Simulation Centre teams designed a Lavender Support facilitator training. Simulation-based education is an effective training modality for building confidence in learners (Cook, 2013). It’s also an opportunity to practice, in a psychologically safe environment, while monitoring staff reactions for signs of a trauma response. The CAMH Lavender Support training involves several stages of progressing complexity, leading to learner independence. Training includes didactic teaching, simulation with a simulated participant (actor), shadowing/observation, and coaching. The learning objectives are to: • Determine when the Lavender debrief model should be used, • Distinguish the four stages of the Lavender debrief model, • Describe affective skills required (e.g. active/empathic listening, self-regulation), and • Demonstrate tasks required during the model (e.g. assessing for PTSD).

Results/findings
Lavender Support training will be delivered to four Spiritual Services staff and students per session. Quantitative and qualitative data will be collected through pre- and post-training surveys. These surveys are guided by Moore’s (2009) Outcome Evaluation Framework, gathering data at multiple outcome levels including level 1 (participation), level 2 (satisfaction), level 3 (declarative/procedural knowledge), and level 4 (competence). Changes in knowledge and confidence from pre-to-post will be assessed, as well as intention to change practice, and learners’ overall experience with the training.

Discussion
Evaluation results will highlight how CPD training, incorporating simulation, impacts the skills, and the confidence of Spiritual Services staff as well as prepares them to provide trauma-informed, psychological support to staff and physicians following a crisis event. We hypothesize the training will increase confidence in learners, which supports the implementation of the Lavender Support system and contributes to professional and organizational resilience in responding to crisis events.

Impact/relevance to the advancement of the field of CME/CPD
This training, based on an evidence-based model, contributes to the understanding of how CPD training, involving simulation, can increase skills in facilitating psychologically safe post-event debriefs and contribute to professional and organizational resilience. It will identify best practices and learning modalities used in other CPD/CME contexts. We will make recommendations on what’s needed to design effective CPD training to develop professional and organizational resilience, as well as engage in a discussion on how these strategies can be applied to their local contexts.

30. Leading from the Inside Out: Design and Implementation of a Realistic CPD-led Physician Leadership Development Program

Author

Matt Orr, PhD, Associate Dean for Continuous Professional Development & Strategic Affairs, University of South Carolina School of Medicine

Purpose/problem statement
Despite increasing demands for and expected competency in effective physician leadership, leadership development is not a core part of physician training. Evidence suggests that the most effective leadership programs are delivered on-site in spaced sessions and include ongoing feedback and content that enables participants to transfer learned knowledge and skills into practice in their immediate work contexts. However, it is common for health systems to send their leaders to external off-site or self-administered programs for their continuing leadership education. While many of these programs provide critical knowledge, they often do not enable participants to transfer that knowledge into practice in their home institutions.

Approach(es)
The Physician Leadership Institute (PLI) is an organization-sponsored internally designed and delivered program at the University of South Carolina School of Medicine Columbia—Prisma Health Midlands. The program engages a small cohort of physician leaders (10-12) in learning and immediately applying evidence-based leadership skills and context-specific practices across nine classroom sessions, monthly individual coaching, and formal leadership assessments over the course of 10 months. Each session prioritizes skills-based practice with feedback and experiential and reflective learning. Participants also completed an improvement project within their area of responsibility, which included intentional application of program content.

Findings
Participants complete in-depth reflection activities throughout the PLI, including end-of-session and comprehensive mid-program, end-of-program, and 9-month post self-evaluations/-assessments. Through these evaluations participants demonstrate the practice of greater self-awareness, self-management and insight into their behavior and the impact of their leadership on others. Participants also describe improved confidence in their leadership abilities, including “leading up” in relation to their senior leaders and more effectively leading change management efforts.

Discussion
We have successfully developed and implemented a CPD-led organization-sponsored physician leadership development program that is rooted in evidence-based best practice and addresses needs relevant to the participants’ immediate work. Participants’ projects not only enabled them to immediately practice change management and other skills, but also addressed an area of departmental or system need for improvement. The second cohort of this program is underway and our partners at the USC School of Medicine Greenville are simultaneously conducting a tailored version of this model.

Barriers/facilitations
Given the time and intensity required to deliver proper leadership development, it is often difficult for more than a few physicians to engage in such programs at a level deep enough to affect an organization in a meaningful way. To overcome this barrier, we worked with senior clinical and operational leaders to design a format and content that they could not only support but also enabled them to set the program as a priority for participation by selected physicians.

Impact/relevance to the advancement of the field of CME/CPD
The PLI serves as a model for a physician leadership program that is both realistic in structure and provides value to physicians and the organization.

31. Addressing Obesity Bias in Healthcare: Development and Testing of an Online Course

Author(s)

Randa M. Kutob, MD, MPH, Associate Professor, University of Arizona, College of Medicine, Tucson

Stephanie Trofymenko, DO Candidate, Medical Student, Noorda College of Osteopathic Medicine

Amit Algotar, MD, PhD, FOMA, Associate Professor, University of Arizona, College of Medicine, Tucson

Background/purpose/inquiry question
Obesity exists as part of a continuum that is often preceded by social determinants of health and results in health disparities with downstream effects in a myriad of chronic diseases. Despite high rates of obesity, studies have documented that health providers lack of confidence regarding obesity treatment and that obesity bias exists in healthcare settings. This study describes the development and preliminary testing of an online, interactive module, focused on obesity, obesity bias, and motivational interviewing. The authors hypothesized that completion of this intervention would result in decreased self-reported obesity bias using a validated measure.

Theoretical framework(s)
The authors created an interactive, case-based online CME learning module, “Counseling Patients About Obesity”. Course content was informed by Adult Learning Theory and Competency-based Medical Education.

Methods
The course’s impact on obesity bias reduction was tested on 103 first year medical students who completed the validated Anti-fat Attitudes Questionnaire (AFA) pre- and post- completion of the online module. This validated measure examines explicit attitudes towards people with obesity and obesity in general. All items were measured on a 9-point Likert-type response format. Higher scores indicated stronger anti-obesity attitudes. The AFA individual domain (Dislike, Fear of Fat and Willpower) mean scores were the primary study endpoints. Paired t-tests were used to compare changes in domain scores for each group between baseline and post-testing. All analyses were performed with IBM SPSS,   Version 28.

Results/findings
A majority of participants identified as female (53%). White/Non-Hispanic (47%) and Hispanic (18%) were the most commonly identified racial/ethnic background. AFA mean domain scores decreased significantly indicating a decrease in explicit anti-obesity attitude bias after completing the online module. This decrease was present in all 3 domains of Fear (Pre-AFA Mean Score = 4.63, Standard Deviation (SD) = 2.34 versus Post-AFA Mean Score = 3.72, SD = 2.60, p < .001); Dislike (Pre-AFA Mean Score = 1.25, SD = 1.30 versus Post-AFA Mean Score = 0.88, SD = 1.15, p < .001); and Willpower (Pre-AFA Mean Score = 3.23, SD = 1.90 versus Post-AFA Mean Score = 2.23, SD = 1.87, p < .001)

Discussion
Preliminary testing of an online, interactive CME course on obesity bias awareness and patient counseling utilizing 1st year medical students as study participants demonstrated promise in the course’s ability to reduce participants’ explicit obesity bias as measured on a validated instrument. Future studies will explore the course’s impact on practicing physicians.

Limitations
The limitations to this study include a single group pre-post design without a control group, and the possibility of social desirability bias affecting answers to the AFA.

Impact/relevance to the advancement of the field of CME/CPD
Although obesity rates have been continually increasing in the United States, studies have reported a decline in addressing weight during clinical visits. For practicing physicians, online courses with interactive components may provide an opportunity to increase bias awareness and safely practice new skills such as motivational interviewing.

32. The Interplay of Resilience, Self-Efficacy, and Organizational Trust

Author(s)

Dillon Welindt, Doctoral Student, University of Oregon

Miranda McDaniel, BS, Research Assistant, Professional Renewal Center

Betsy W. Williams, PhD, MPH, FSACME, Clinical Program Director, Professional Renewal Center

Background/purpose/inquiry question
The importance of individual and organizational resilience in the medical industry is well-understood, albeit more so following system stressors related to COVID-19. In the aftermath of the pandemic, a clear imperative has arisen to identify determinants of individual and organizational resilience to ensure the ability of medical organizations to respond effectively to emergent demands.

Theoretical framework(s)
This work takes the perspective that a resilient organization requires resilience at the individual (their ability to effectively navigate setbacks) and organization level (remaining connected to the system’s mission in the face of challenges). The research goal is to elucidate those relationships in aid of informing future CME/CPD programming.

Methods
Chiefly, data stem from a sample of physicians and support staff (n=65) at a small midwestern medical center that engaged these authors to assess and improve quality. Supporting analyses are drawn from a sample of physicians and trainees (n=415) referred to a midwestern center secondary for assessment and/or treatment secondary to professionalism lapses. The core analysis of this work is a structural equation model testing the relationship among self-efficacy, resilience, burnout, and organizational trust. This model was developed in part using information response theoretic techniques to develop a simplified but valid model. Factor analyses were employed to identify salient dimensions in the measures used.

Results/findings
Findings indicate a complex relationship among the aforementioned constructs. Consistent with the broader literature, findings support the centrality of self-efficacy in the context of choice and suggest a nuanced perspective on resilience and self-efficacy. Additional effects of wellbeing were found.

Discussion
While exploratory, they indicate that resilience partially mediates the effect of self-efficacy on organizational connectedness (the respondent’s sense of their engagement with the team, their patients, and the broader system). One explanation for this would be that over repeated challenges, resilience becomes increasingly important in a decision to engage or not. Their sense of connectedness, consequently, is associated with the trust they put in their organization.

Limitations
These data may not generalize to the broader physician population. Casual inferences from SEM are debated.

Impact/relevance to the advancement of the field of CME/CPD
These findings highlight the interplay between the individual and the broader system in fostering organizational resilience, demonstrating a probable nexus between individual and system-level resilience. However, the broader literature has unequivocally demonstrated the effects of individual/system mismatch, excessive workloads, and lack of autonomy as having negative consequences on the individual’s resilience, self-efficacy, and wellbeing. Study findings suggest those individual consequences may be part of a feedback loop to organizational wellbeing. This would be in accordance with scholarship on management. With regard to the medical educator tasked with fostering individual and organizational resilience, this work makes a few key points: 1. Diminished individual self-efficacy and resilience are associated with increased burnout (and, consequently, lower patient empathy). 2. Organizational trust and engagement are also associated with individual well-being, self-efficacy, and resilience. 3. Therefore, the aims of improving empathy, fostering individual resilience, and building organizational resilience, are highly related. 4. A CME/CPD activity targeting organizational resilience should consider the individual characteristics of the system’s providers, as well as the unique culture of that system.

33. Comparison of Two QI Approaches to Improve Glycemic and Weight Outcomes in T2D in Underserved Populations

Author(s)

Natalie Sanfratello, MPH, CHCP, Senior Program Manager – Quality Improvement, Educational Programs, and Contracts, Boston University Chobanian & Avedisian School of Medicine, CCE

Caroline Pardo, PhD, CHCP, FACEHP, President, CPD, Clinical Education Alliance

Linda Baer, MSPH, CHCP, Grant and Diabetes Education Manager, Boston University Chobanian & Avedisian School of Medicine, Center for Continuing Education

Erika Brechtelsbauer, PharmD, Director, Educational Strategy, Clinical Care Options

Robin Black, PharmD, BCACP, Scientific Director, Clinical Education Alliance

Julie White, MS, FSACME, CE Director, Boston University Chobanian & Avedisian School of Medicine, CCE

Background/purpose/inquiry question
The aim of our QI initiative was to improve glycemic control and weight management in patients with T2D through distinct approaches in 2 care settings, Boston Medical Center (BMC) and Federally Qualified Health Centers (FQHCs). Our primary research aim was to test the effectiveness of QI methods to improve the percentage of patients with T2D who have an A1c < 9% and, secondarily, the intervention effectiveness within varying settings.

Theoretical framework(s)
Through a lens of constructivism, the research team sought to determine whether an under-resourced FQHC could develop capacity to conduct QI initiatives through remote mentoring by an experienced QI team. The research aims to explore the factors related to capacity development within care settings.

Methods
From March 2022 through July 2023, QI projects using the IHI’s Model for Improvement were conducted at 3 sites with the same aim to increase the % of patients with T2D who have an A1c < 9% by 7/31/23. At BMC, the approach involved high-touch project management and expertise embedded in the clinic. At the FQHCs, a remote mentoring program designed to build capacity for the clinics to conduct projects. Each site tailored interventions based on the resources and needs in their care settings. BMC implemented interventions to improve appropriate prescribing of GLP-1 RAs and SGLT2is through a suite of interventions. One FQHC implemented nutrition-based interventions and improved processes for health maintenance management, and the other FQHC implemented patient education, workflow improvement for clinical pharmacists, and increased use of a health maintenance tool.

Results/findings
At FQHC1, the % of patients (N = 165) with an A1c< 9% increased from 75.6%-83% as of 6/1/2023. At FQHC2, the % of patients (N = 644) with an A1c< 9% decreased from 68.1%-60.4% as of 8/1/2023. At BMC, the % of patients (N = 7647) with an A1C< 9% increased from 76.3%-78.8% as of 8/1/23. A qualitative analysis of the barriers experienced by site teams during the implementation of the interventions yielded findings related to team structure, such as lack of effective and frequent communication, lack of on-site intervention champion support, and lack of sufficient staffing to support clinic tasks.

Discussion
QI initiatives can be effective in closing the gap between current practice and best practices, but interventions must be tailored to the needs and resources of each care setting. Both settings, have seen some measure of improvements despite having undergone different approaches related to the amount of support received from the QI expert team. The repetition built into QI processes (e.g., PDSA cycles) creates QI sustainability due to the deep learning that allows participants to carry forward their QI skills. These processes can be effectively taught in-person or remotely.

Limitations
Each site was limited by the resources and leadership support available to them. Although both FQHCs received the same mentorship and instruction, one faced more challenges due to staff/leadership turnover and competing priorities.

Impact/relevance to the advancement of the field of CME/CPD
Remote mentoring on quality improvement projects can build capacity within under-resourced sites to further clinician performance and patient outcomes

34. Improving Interprofessional Empathy in Physicians Referred for Remediation

Author(s)

Nathaniel Williams, Student, Manager, Student, Research Assistant, Professional Renewal Center

Dillon Welindt, Doctoral Student, University of Oregon 

Michael Williams, PhD, Principal, Wales Behavioral Assessment 

Betsy W. Williams, PhD, MPH, FSACME, Clinical Program Director, Professional Renewal Center

Work in progress

Background/context/inquiry question
Empathy has been recognized to be an important aspect of health care delivery 1. While empathy is generally recognized to be a broad, multidimensional construct, its discussion in the medical literature more narrowly operationalizes it in terms of the physician-patient relationship 2. There is comparatively little discourse in the medical literature on empathy across healthcare professionals, despite empathy being recognized as an important predictor of workplace effectiveness and prosocial behavior 3. Both ACGME and CanMEDS have identified the importance of communication within their Core Competency areas for medical providers. 4. Empathy, through various paths, is associated with effective communication, and has been identified as especially important in workplaces that are intercultural or interdisciplinary, attributes that increasingly describe the medical environment. Thus, developing empathy in medical professionals would be of great importance, particularly in those who demonstrate competence in communications-related domains. This work seeks to assess the outcome of an educational intervention aimed at improving interprofessional empathy, and correlates of successful outcomes therein.

Methods
Data will be drawn from a sample of physicians engaged in a CME activity designed to support their professional development in the areas of Interpersonal & Communications Skills, Professionalism, and Systems-Based Practice. These physicians had previously presented with professionalism lapses, largely comprised of communications issues. These same participants will engage in an educational intervention aimed at improving interprofessional empathy. This is a web-based intervention, occurring in 90-minute didactic and interactive sessions. The interprofessional empathy development of the study group will be tracked using a psychometrically validated scale of empathy. A general linear model will be used to test mean differences over time, as well as to identify significant predictors of empathy change.

Impact/relevance to the advancement of the field of CME/CPD
These data will provide evidence (or lack thereof) for an educational intervention to improve interprofessional empathy in physicians referred for remediation. As well, these data could provide valuable insight into the empathy development process for educators and CME stakeholders. Communication has been identified consistently as a core competence in medical practice. Given the central role of empathy in communication, this is a valuable endeavor.

Preliminary Findings
This is a work in progress; data collection is pending; analyses will follow.

Reference to current literature/perspective on the topic
1. Hemmerdinger JM, Stoddart SD, Lilford RJ. A systematic review of tests of empathy in medicine. BMC Med Educ. 2007;7(1):24. doi:10.1186/1472-6920-7-24 2. Pedersen R. Empirical research on empathy in medicine—A critical review. Patient Educ Couns. 2009;76(3):307-322. doi:10.1016/j.pec.2009.06.012 3. Clark MA, Robertson MM, Young S. “I feel your pain”: A critical review of organizational research on empathy. J Organ Behav. 2019;40(2):166-192. doi:10.1002/job.2348 4. Fuller M, Kamans E, Van Vuuren M, Wolfensberger M, De Jong MDT. Conceptualizing Empathy Competence: A Professional Communication Perspective. J Bus Tech Commun. 2021;35(3):333-368. doi:10.1177/10506519211001125

35. Utilizing Web Delivered Simulations to Determine Practitioner Communication Through Communication Styles and Guide Learning Process

Author(s)

Rodderick L. Williams, Research Assistant, Wales Behavioral Assessment

Michael Williams, PhD, Principal, Wales Behavioral Assessment

Betsy W. Williams, PhD, MPH, FSACME, Clinical Program Director, Professional Renewal Center

Work in progress

Background/context/inquiry question
There is considerable research on the importance on communication in healthcare. There are two, somewhat distinct, literatures. The first is on effective communications with patients; the second is the effective communications within the medical team. The consequences of incomplete or ineffective communications between healthcare team members is well documented and methods to ensure that communications have been effective and accepted have been published. However, more informal/naturalistic communications between physicians and members of their support staff/healthcare team are less explored. One possible reason for this is the difficulty in assessing communications style. Four to five communication styles are typically cited passive; passive aggressive; aggressive; assertive; and manipulative.

Reference to current literature/perspective on the topic
We developed a purpose-built web simulation that allows us to determine the preferred communication style of physicians. We aim to better understand how various styles of physician communication support collaborative interactions in aid of facilitating more effective communication between a physician and their team members.

Theoretical framework(s)
In our experience an impediment to effective healthcare communication is that the speaker often does not have insight into their communication style or impact of that style on the receiver. This is consistent with the broader literature on lack of insight for underperformance (Dunning and Kruger effect and Hays, et al 2002). A potential way to overcome such a limitation is to assist the individual in assessing their actual performance. We developed a communication exercise that meets this need. To increase the tool’s utility, the task requires cooperation and provides a performance stress on the communicator.

Methods
We are testing a 7–8-minute Web delivered fuzzy logic simulation. The Web model will provide data to analyze reactions and communication styles of the participants. The data allow us to display physician’s communication style versus alternative styles which can be shared to help exemplify recommended communication for them.

Impact/relevance to the advancement of the field of CME/CPD
The relevance of a valid, inexpensive, and short assessment of physician’s dominant communications style would provide the basis of educational training and intervention to improve both the physician’s and the team’s functioning.

Preliminary Findings
The low fidelity simulation emulates stressful environments. The simulation program gathers data on the pattern of choices made by the participant in order to evaluate their communication style. This is used to help further communications with the participant. Descriptive findings as well as the model itself will be provided and discussed.

36. Framework and practical strategies for cooperation across multiple stakeholder groups for the development of a national CPD ecosystem

Author(s)

Grahanya Sachidanandan, BHSc, Medical student, University of Toronto

Abhimanyu Sud, MD CCFP, Research Chair, Primary Care & Population Health Systems, Humber River Hospital

Purpose/problem statement
In the context of a worsening overdose crisis and rapidly changing evidence and clinical guidance related to pain management, there has been welcome growth in the number, reach, and maturity of pain-related continuing professional development programs for health care providers across Canada. These programs aspire to impact patient and population health outcomes by improving pain-related clinical outcomes, access to high quality pain care, and effective opioid prescribing. Despite this expansion of programs and some increasing informal cooperation between them, gaps continue to exist in access to accredited pain CPD programs and uncertainty remains as to their higher-level impacts. Through the conceptualization and initiation of a national pain CPD ecosystem, this project explored how the deliberate connection of stakeholder groups can address gaps in the current landscape and better equip the system to maintain effectiveness in the face of continual evolution.

Approach(es)
To foster the development of a national ecosystem, we identified a comprehensive list of 30 contemporary pain CPD programs from across the country and targeting multiple professions. To understand how the programs work and initiate deliberative dialogue, the team conducted structured interviews with 27 program leaders. Findings were integrated and reported using a theory-based mechanistic model for CPD, and dialogue was advanced through two large group meetings: one focused on reporting and collaboration between 25 program leaders and one focused on system planning with 11 national pain CPD leaders.

Findings
The majority of programs operate in independent siloes, resulting in a significant burden on individual program planners, administrators, and educators to coordinate funding, accreditation, resource development, recruitment, program delivery, and evaluation. As such, the resounding consensus of our multistage deliberative dialogue was the need and eagerness for cooperation – not only between programs but also with other stakeholder groups, including policymakers, institutions, learners, and patients.

Discussion
The recommendations and strategies to facilitate cooperation arising from our interviews and meetings can be systematized and communicated as a framework for multistakeholder collaboration for the development of a dynamic and resilient national CPD ecosystem.

Barriers/facilitations
The framework outlines practical, evidence-based strategies for multilevel cooperation and engagement including government cooperation to facilitate policy engagement, institutional cooperation to facilitate training in practice settings, program cooperation to facilitate resource sharing, learner cooperation to facilitate deliberate CPD engagement, patient cooperation to facilitate high relevance and impact, and scholarly engagement to facilitate evaluation. This framework has the capacity to address the many reported barriers to providing effective and sustainable pain CPD, including variable participation, unstandardized competencies, and lacking evaluation capacity.

Impact/relevance to the advancement of the field of CME/CPD
CPD programs are challenged to produce sustained systemic change when operating in isolation, which is further challenged by the constantly evolving clinical, evidentiary, policy, and institutional environments in which these programs operate. This framework supports not only cooperation between existing programs, but also collaboration at policy, clinical, academic, learner, and patient levels. The strategies contained therein aim to alleviate and distribute some of the burden on individual CPD planners, mitigating burnout and increasing the capacity of educators and institutions to respond to evolving contexts.

37. Curricular Co-construction: Collaborating to Enhance Child and Adolescent Psychiatry Training in Equity, Diversity, and Inclusion Principles

Author(s)

Nikhita Singhal, MD, Psychiatry Resident, University of Toronto 

Jenny Chum, MD, Psychiatry Resident, University of Toronto

Catherine Deschênes, MD, Psychiatry Resident, University of Toronto

Ayan Dey, MD, PhD, Psychiatry Resident, University of Toronto

Oshan Fernando, PhD, Clinical Research Project Coordinator, The Hospital for Sick Children 

Arfeen Malick, MD, Staff Psychiatrist, The Hospital for Sick Children 

Jude Sanon, MD, Psychiatry Resident, University of Toronto 

Yezarni Wynn, MD, Psychiatry Resident, University of Toronto 

Raj Rasasingham, MD, Staff Psychiatrist, University of Toronto

Chetana Kulkarni, MD, Staff Psychiatrist, University of Toronto 

Work in progress

Background/context/inquiry question
The current social climate has brought attention to longstanding systemic inequities impacting youth mental health. However, equity, diversity, and inclusion (EDI) principles have not been a major component of Canadian child and adolescent psychiatry (CAP) training. Our objective is thus to co-design and develop an innovative, evidence-informed educational intervention addressing this gap with the ultimate aim of promoting reflective practice and helping clinicians provide more compassionate and holistic care.

Reference to current literature/perspective on the topic
Work done about underserved and marginalized populations should not be completed without their input and recognition. Additionally, education addressing underserved populations is more effective when learning encourages critical reflection, ensures understanding of health determinants from a trauma-informed, anti-oppressive structural lens, and personalizes the experiences of the populations of interest.

Theoretical framework(s)
Our study is based on Kern’s six-step approach to curricular development, following principles of design theory by defining the intervention after performing a needs assessment (then using this process to iteratively design, test, and evaluate the modules developed using Kirkpatrick’s Program Evaluation Model).

Methods
We began with an environmental scan to better understand the current state of CAP EDI training in Canada, surveying program directors, current trainees, and recent graduates using online questionnaires and semi-structured follow-up interviews. We then engaged youth advisors with lived experience to co-design and develop a series of online learning modules addressing educational gaps identified through our needs assessment. The modules will be tested, evaluated, and iteratively fine-tuned based on feedback from participants completing online self-assessment surveys immediately upon completion of each module and again three months later.

Impact/relevance to the advancement of the field of CME/CPD
EDI principles are an essential component of any CME/CPD activity and involving individuals with lived experience in the development of such activities is vital. Our combined process- and outcome-oriented approaches will support CAP EDI postgraduate training as well as lifelong EDI self-evaluation for CAP clinicians, and we anticipate this may serve as a foundation to be adapted for broader applicability to a variety of medical specialties and interdisciplinary health care professionals.

Preliminary Findings
For our environmental scan, we received a total of 31 survey responses and conducted 3 semi-structured follow-up interviews. The data gathered indicated a significant gap in EDI training across programs, with 71% of respondents perceiving either a moderate or major gap in training. Identified barriers included soliciting local expertise and finding space/time within curricula. All program directors were interested in access to externally developed resources. Five specific areas of need identified were cultural formulation, anti-Black racism, Indigenous issues, LGBTQ+ populations, and refugee mental health — these are the topics for the five planned modules in the series. We have developed the first two modules and piloted Module 1 (Cultural Formulation) among CAP residents across the country. Seven participants completed the post-module questionnaire; all respondents reported having gained new skills through completion of the module and the vast majority (86%) reporting they intended to make changes to practice as a result. We continue to apply constructive feedback received to help inform module revisions and iterative improvements, as well as development of the remainder of the modules in the series.

38. Clinician Behavior Change Around Judicious Antibiotic Prescribing for Pediatric Community Acquired Pneumonia 

Author(s)

Sharisse Arnold Rehring, MD, FAAP, Director, Medical Education CPMG, Clinical Professor University of Colorado School of Medicine, Director, Medical Education, Director Pediatric Education, Colorado Permanente Medical Group 

John Steiner, MD, MPH, Senior Researcher, Institute for Healthcare Research, Colorado Permanente Medical Group

Matthew Daley, MD,, Senior Researcher, Institute for Healthcare Research, Colorado Permanente Medical Group

Background/purpose/inquiry question
Antibiotic stewardship preserves the effectiveness of common antibiotics and protects patients from unnecessary treatment. To promote judicious antibiotic use for children with community-acquired pneumonia (CAP), we delivered two educational sessions, 13 months apart, that included interactive, case-based CME with new prescribing tools in the electronic health record (EHR). The sessions encouraged use of narrow-spectrum antibiotics and reductions in duration of treatment. Participants were introduced to a new order set in the EHR that facilitated guideline-concordant prescribing. We evaluated these programs to determine whether the: 1) first session changed clinician prescribing behavior; and 2) second session further reinforced changes in prescribing behavior.

Theoretical framework(s)
Longitudinal programs on the same topic can increase CME effectiveness, but the additive effect on clinician behavior change is uncertain. Whether simultaneous EHR tools reinforce behavior change in conjunction with an educational intervention is another important question.

Methods
The study took place in Kaiser Permanente Colorado (KPCO), an integrated healthcare system that employs full-time, primary care physicians, nurse practitioners and physician assistants to care for children. Diagnoses of CAP, antibiotic prescriptions, and identifiers for prescribing clinicians were drawn from EHRs. CME attendance was identified from the KPCO learning management system. Study outcomes included the choice of a narrow-spectrum antibiotic and prescription duration. CME interventions took place on September 22, 2020 and October 26, 2021. We defined four time periods: pre-intervention (1/1/2016-3/14/2020); early COVID-19 (3/15/2020-9/22/2020); post-intervention (9/23/2020 – 10/26/2021); and reinforcement (10/27/2021- 10/26/2022). We excluded the early COVID-19 period because of multiple changes in respiratory illness patterns, care delivery site and utilization. We conducted interrupted time series analysis to assess changes in antibiotic choice and duration across the three remaining time periods.

Results/findings
240 clinicians (physicians, nurse practitioners, and physician assistants) attended the first CME and 278 clinicians attended the second CME session. We identified 3570 cases of community acquired pneumonia during the entire observation period. In children diagnosed with CAP, prescribing of narrow-spectrum antibiotics increased from 41% pre-intervention to 68% post-intervention and 69% in the reinforcement period (p< 0.001). For children and adolescents with an initial narrow spectrum antibiotic, duration decreased from the pre-intervention (mean duration 9.9 days, standard deviation [SD] 0.5 days) to post-intervention (mean 8.2, SD 1.9) to reinforcement (mean 6.8, SD 2.3) periods (F-test p< 0.001). A pneumonia order set in the EHR was used for 33% of prescriptions during the post-intervention period and 66% of prescriptions during the reinforcement period; there was no pediatric pneumonia order set during the pre-intervention period.

Discussion
Initial improvements in antibiotic choice and duration for children with CAP were sustained for over 24 months after a CME session. A reinforcement CME session further reduced the duration of prescribing but did not have additional effects on antibiotic choice.

Limitations
Limitations of the study included the small sample size of clinicians and the setting of the study in a highly integrated healthcare system.

Impact/relevance to the advancement of the field of CME/CPD
Repetition of critical CME content and the use of EHR-based prescribing tools may maintain or reinforce changes in clinician behavior.

39. Promoting Compassion in Healthcare Ethics Consultation Education: A Framework to Approach Structural Stigma in Mental Health and Substance Use Health

Author(s)

Zahra S. Hasan, MHSc, Research Analyst, Centre for Addiction and Mental Health

Daniel Z. Buchman, PhD RSW, Bioethicist and Scientist, Centre for Addiction and Mental Health

Purpose
The ability for healthcare ethicists to approach ethical issues in mental health and substance use health (MHSUH) skillfully is essential given that they are highly stigmatized conditions and are intrinsically linked to domains of clinical practice, organizational policy, and public health. To our knowledge, there is no educational framework in the published literature that explores competency-building of structural stigma and MHSUH for healthcare ethics consultation (HCEC). In this project, we address the need for, and ultimately propose, an educational framework to support competency-building of structural stigma in MHSUH for HCEC learners.

Approaches
To guide our framework, we applied concepts of structural stigma, structural competency, structural humility, transformative learning, and dialogic learning. We conducted a literature search using OvidMEDLINE and Google Scholar to identify and review relevant peer-reviewed and grey literature in the areas of bioethics and the health professions, education sciences, stigma studies, and MHSUH. A total of n = 123 records were included in analysis (OvidMEDLINE n = 81, Google Scholar n = 38, conference presentation n = 1, grey literature n = 3). We summarized the data inductively and consolidated key themes to inform a preliminary draft framework for review by content experts. Then, we conducted 30-minute quality improvement (QI) interviews with 6 educator-perspective participants (e.g., ethicist, stigma scholar, mental health clinician) and 1 learner-perspective participant (e.g., Fellow with no specialized training in MHSUH) to obtain feedback on the draft framework’s strengths, limitations, and utility.

Findings
We identified the following themes in the literature, which we used to inform our proposed key concepts for the draft framework: compassion, critical reflection, humility, and interprofessional practice. QI interview participants resonated strongly with these proposed key concepts. Participants noted limitations (e.g., lack of empirical evidence and examples) and the potential for our key concepts to support (1) other clinical learners and (2) other patient populations who historically experience structural stigma (e.g., people living with intellectual and development disabilities).

Discussion
Our proposed framework is comprised of three key concepts, intended to be applied in tandem, to meaningfully support competency-building to address structural stigma in MHSUH for HCEC:

1.         Integrate self-reflexive practices into formal, informal, and hidden curricula

2.         Embed structural humility into multiple teaching methods and contexts of learning

3.         Balance critical consciousness and compassion in dialogue

Barriers/facilitations
As described in the literature and our QI interviews, evaluating values-laden competencies is challenging, especially those that require vulnerability on behalf of the learner. Research suggests that micro- and meso- level interventions, alone, are not sufficient methods to address MHSUH structural stigma. This framework is intended to promote competency-building and awareness of issues that exist at the macro-level.

Impact to the advancement of the field of CPD
Our proposed framework can equip HCEC learners with the knowledge and awareness needed to help address the larger structural issues within the systems they serve. CPD on MHSUH structural stigma for HCEC can support compassionate ethical decision-making processes for all who engage with the healthcare system (e.g., patients, clinicians, policy makers, etc.).

40. Supervisor Experiences Using an R2C2 Model of Feedback that Encourages Reflection on Power and Intersectionality

Author(s)

Shaheen A. Darani, MD, FRCPCDirector, Faculty Development & Assistant Professor, Dept of Psychiatry, Associate Director, PG Wellness, Temerty Faculty of Medicine, University of Toronto

Alissa Liu, MD, Psychiatry Resident, University of Toronto 

Anupam Thakur, MBBS, MD, MSc, Staff Psychiatrist, University of Toronto Department of Psychiatry, Education Scholar, CAMH

Ivan Silver, MD, FRCPC, Psychiatrist and Full Professor, University of Toronto

Csilla Kalocsai, PhD, Assistant Professor, Department of Psychiatry, University of Toronto

Sanjeev Sockalingam, MD, MHPD, FRCPC, Chief Medical Officer, Vice-President Education, and Senior Scientist, Professor, Department of Psychiatry, CAMH and University of Toronto Department of Psychiatry 

Sophie Soklaridis, PhD, Scientific Director and Senior Scientist, Center for Addiction and Mental Health

Background/purpose/inquiry question
Feedback is a teachable skill that is increasingly important in medical education with CBME. The R2C2 model is a theory-informed, evidence-based approach to providing feedback. Unexamined power affects the effectiveness of feedback provided to trainees. The literature shows that evaluators face difficulties assessing competency independent of learner’s gender identities. Female trainees receive more inconsistent feedback compared to males and less supportive reference letters. With respect to the impact of race on feedback, a study by Cohen and colleagues showed that black students rate higher levels of evaluation bias than white students in response to critical feedback. This highlights the need to consider intersectionality when providing feedback. The literature shows many faculty struggle to engage with these concepts. Approaches that consider how intersectionality impacts trainees and faculty are key to reducing bias in feedback. To date, the R2C2 model has not been studied through the lens of power within a supervisor-trainee relationship. This study explores supervisors’ experiences using an R2C2 model, that encourages reflection on intersectionality.

Theoretical framework(s)
The R2C2, an evidence based and theory informed feedback model, was a integral part of the study.

Methods
An exploratory research design using qualitative methods was used. Psychiatry resident supervisors received faculty development on the R2C2 model, racial and gender bias, and intersectionality, and were encouraged to use this in feedback discussions. Ten supervisors participated in semi-structured interviews. They comprised five questions that aimed to elicit supervisors’ experiences of using R2C2, and their reflections on intersectionality. Probes were used for participants to comment on the impact of gender, race or other social categories in the supervisor-resident relationship. This interview questions on power dynamics were key vehicle through which the concept of intersectionality was introduced to supervisors in addition to training. Interviews were audio-recorded, transcribed, and analyzed using a thematic approach.

Results/findings
Qualitative data analysis revealed four key themes: When we are similar, it’s business as usual; Power is implicit so we need to think about it; “Just because I’m a woman, don’t expect me to be”; and Power is assumed so we don’t need to talk about it.

Discussion
Supervisors had mixed views about the value of reflection on intersectionality in feedback. Possible explanations for this finding relate to the influence of individual experiences, assumptions, and biases; varied responses to change; and differing levels of supervisor comfort. Further exploration of the impact of intersectionality and power dynamics on feedback processes is needed. Findings suggest a need for faculty development/CPD to establish skills in navigating these complex conversations in the supervisor-trainee relationship. Future research should focus on fidelity measures and resident voice in the evaluation, something we are not able to explore in this pilot project.

Limitations
Limitations include the single institution design and the omission of resident experience of supervisors using the R2C2 model of feedback that encourages reflection on power and intersectionality.

Impact/relevance to the advancement of the field of CME/CPD
This study is one of the first to explore intersectionality in feedback and offers insights for faculty development related to the diverse perspectives that supervisors hold. Findings from this study could inform future faculty development/CPD programs.

41. Harnessing the Power of Natural Language Processing in CPD Program Evaluation: Findings from a De-escalation Training Program for Healthcare Professionals in a Tertiary Mental Health Facility

Author(s)

Anupam Thakur, MBBS, MD, MScCAMH, Staff Psychiatrist, University of Toronto Department of Psychiatry, Education Scholar, CAMH

Kayle Donner, MA; M.Ed., Research Methods Specialist, Center for Addiction and Mental Health (CAMH)

Faisal Islam, PhD, CAMH, Manager, Education, Evaluation and Quality Improvement, Center for Addiction and Mental Health (CAMH)

Kenya A. Costa-Dookhan, MSc, Medical Student, University of Toronto Temerty Faculty of Medicine 

Sanjeev Sockalingam, MD, MHPE, FRCP(C), FACLP, Chief Medical Officer, Vice-President Education, and Senior Scientist, Professor, Department of Psychiatry, CAMH and University of Toronto Department of Psychiatry 

Marta Maslej, PhD, Staff Scientist, Krembil Centre for Neuroinformatics, Center for Addiction & Mental Health (CAMH)

Work in progress

Background/context/inquiry question
There is emerging literature on the use of artificial intelligence applications such as natural language processing in medical education. However, few studies have explored its role in program evaluation. This project aims to explore innovative ways to integrate natural language processing in CPD program evaluation.

Reference to current literature/perspective on the topic
Similar to healthcare applications, the role of artificial intelligence (AI) in health professions education is being increasingly recognized. Natural language processing (NLP), a branch of AI, can be a valuable resource in analysis of large educational datasets. Although it provides rich opportunities to understand feedback from educational programs, few studies have explored its role in education program evaluation. This study explores the use of natural language processing in program evaluation of Trauma-Informed De-escalation Education for Safety and Self-Protection (TIDES), a de-escalation training program for healthcare staff and physicians in a large tertiary mental health facility.

Possible theoretical framework(s)
The innovative methods used in the project utilizes Natural Language Processing to Information Retrieval (NLPIR), a theoretical framework to integrate natural language processing applications (Zhou and Zhang, 2003)

Possible methods
Dataset consisted of post-TIDES training survey responses from 481 participants, including their gender, time spent in their current role, rated satisfaction with the training on a scale from 1-5, as well as two unstructured text responses, reflecting on intent to apply training and open-ended feedback. Topic modeling, specifically Latent Dirichlet Allocation (LDA) and BERTopic, was used to uncover themes in the unstructured responses. Sentiment analysis (SentimentR) was used to assess the valence of open-ended feedback (i.e., the degree to which it was positive or negative).

Impact/relevance to the advancement of the field of CME/CPD
The findings can help understand the role of natural language processing in program evaluation. The innovative methods proposed in the study can be used in program evaluation of future CPD programs.

Preliminary Findings (if any)
Overall, there was little to no distinction in the content of topics emerging from the analysis. Analysis of open-ended feedback suggests sentiment scores were similar based on mean averages and down-weighted zero averages (M=0.42, SD=0.35), ranging from -0.62 to 1.62. Responses were skewed toward positive sentiment, with 249 of 291 (86%) responses obtaining a sentiment score > 0. Mean sentiment scores tended to be higher at higher ratings of satisfaction, but little variability in satisfaction ratings limited our ability to validate sentiment scores. Exploratory analyses using BERTopic yielded intelligible clusters of responses sharing similar meanings. The findings from the study suggest natural language processing tools (i.e., BERTopic) can be used to understand program evaluation feedback. Relatively short texts in open-ended feedback, small data-set and absence of manual categorization of data, may have limited the interpretation of findings derived with sentiment analysis and LDA. The lack of term diversity in LDA suggests pre-trained large language models may be a more promising approach to analyzing feedback. This needs sustained collaboration between AI and program evaluation experts. Future studies with improved survey design, larger data sets and appropriate AI tools are needed to realize the power of NLP in improving training quality.

42. Use of Social Media for Enhancing Virtual Cardiovascular Grand Rounds at University of Oklahoma

Author

Chittur A. Sivaram, MD, Associate Dean for Continuing Professional Development, University of Oklahoma

Purpose/problem statement
Underutilization of social media to promote virtual regularly scheduled series (RSS) is a missed opportunity in the field of CME. In addition to reaching many younger professionals, social media use can enhance the profile of the entity/institution offering the RSS. Careful planning of the timing and techniques of social media use could provide an opportunity to strengthen post-session deep dive learning. The post-pandemic transition of University of Oklahoma Cardiovascular Grand Rounds to virtual format allowed us to introduce social media use (Twitter/X) both pre- and post-weekly sessions.

Approach(es)
We began announcing our weekly sessions by Twitter/X soon after the pandemic when virtual format was first consistently introduced in this RSS. The session announcement is typically placed on Twitter/X two days prior to the actual session. An additional Tweet is made immediately after the session, summarizing the key take home points from the session. Recently, we have also started to provide links to key articles and/or additional ‘deep dive’ learning resources for our audience.

Findings
Our attempts at social media engagement has been positive. Almost 100% of our announcements get retweeted/reposted, increasing the visibility of our RSS. Our audience has roughly doubled (15-20/session to 35-40/session).

Discussion
Our attempts at use of Twitter/X in connection with our Virtual Cardiovascular Grand Rounds was a successful venture, possible largely because these sessions are exclusively virtual. The long-term positive impact on patient care outcomes by this innovation remains unknown.

Barriers/facilitations
The process of placing posts on social media is time consuming. The post-session posting of important slides from the session requires acquisition of slides from the screen and placing them on Twitter/X. Very rarely, speakers decline to have their slides placed on Twitter/X.

Impact/relevance to the advancement of the field of CME/CPD
1. Greater reach of the session, locally within University of Oklahoma as well as outside to other states in US and rarely abroad. 2. Increased attendance from a younger audience (assuming that they use social media more often). 3. Ability to provide post-session learning material.

43. An Analysis of Coping Strategies Used by Racialized Women Clinicians’ Providing Diabetes Care

Author(s)

Maud Ahmad, BSc, Medical Student, Schulich School of Medicine

Tehmina Ahmed, BSc MD FRCPC MScCH, Assistant Professor, Staff Endocrinologist, Clinician Teacher, University of Toronto

 Arani Sivakumar, MPH, Medical Student, Schulich School of Medicine

Catherine Yu, MD FRCPC MHSc, Associate Professor, Associate Scientist, Staff Endocrinologist, University of Toronto

Background/purpose/inquiry question
Racialized women clinicians (RWCs) face multiple forms of discrimination, including gender and racial prejudice, as they carry out their responsibilities in healthcare settings. The mistreatment they endure limits their professional growth and adds stress to an already demanding job. With this complex backdrop, this study seeks to analyze the coping mechanisms RCWs utilize in managing the emotional and professional challenges that come from such stressors.

Theoretical framework(s)
This study employed a constructivist grounded theory approach. In this approach, interviews were analyzed and quotes were deductively mapped to Endler and Parker’s Coping Inventory for Stressful Situations (CISS). This framework adapted the seminal work of Lazarus and Folkman’s problem-focused and emotion-focused categorization, and developed a three-factor model: task-oriented, emotion-oriented, and avoidanceoriented coping. The psychometric properties of the CISS have since been studied in various healthcare settings, with conclusions suggesting it has robust validity and reliability, as well as in settings of racism, discrimination, and mistreatment.

Methods
Semi-structured interviews were conducted with 24 RCWs. Participants were recruited via convenience and virtual snowball sampling, and included physicians, nurses, social workers, and registered dieticians, in Canadian diabetes care settings. 45 to 60 minute interviews were conducted using semi-structured interview guides. After categorizing interview content into the CISS framework, themes and subthemes were constructed based on patterns in responses and set the stage for meaningful discussion.

Results/findings
We identified four themes: (1) Silence and inaction are coping responses driven by fear of professional consequences and emotional burnout, (2) RWCs transform perceived disadvantages into opportunities without directly confronting mistreatment, (3) Some participants engage in advocacy and direct resolutions with perpetrators of biased and prejudiced mistreatment, and (4) Time and experience impact the evolution of coping strategies, self-efficacy, and advocacy.

Discussion
This study illuminates coping mechanisms employed by RWCs in various healthcare settings. These findings necessitate institutional reforms, including open and safe communication channels and a culture that supports advocacy. Additionally, the results underscore the importance of mentorship programs, suggesting they could benefit young clinicians by providing them with the tools to respond to mistreatment and empowering them to advocate for change.

Limitations
The study primarily focuses on the Canadian healthcare system, limiting its generalizability to international contexts. The convenience and snowball sampling methods, while effective for recruitment, may introduce selection bias. Additionally, our sample size of 24 RCWs, though diverse in professional roles, may not capture the full range of experiences and coping strategies of RCWs across different settings and regions. Finally, the study relies on self-reported data, which can be subject to recall and reporting biases.

Impact/relevance to the advancement of the field of CME/CPD
This study provides valuable insights into the unique challenges and coping mechanisms employed by RCWs in diabetes care settings. By identifying areas where systemic bias and discrimination exist, the research underscores the need for targeted Continuing Medical Education (CME) and Continuing Professional Development (CPD) programs. These programs could focus on cultural competency, allyship, and advocacy skills, thus fostering a more inclusive healthcare environment. Moreover, the findings highlight the benefits of mentorship, suggesting an avenue for future CME/CPD activities aimed at empowering underrepresented clinicians through mentorship initiatives.

44. From Maintenance of Certification to Evolution of Competence

Author(s)

Steven Bellemare, MD CPE, Director, Member Experience and External Relations, The Royal College of Physicians and Surgeons of Canada

Lyn Sonnenberg, MD, MEd, MSc, EMBA, FRCPC, Director, Learning Transformation, The Royal College of Physicians and Surgeons of Canada

Guylaine Lefebvre, MD, Executive Director, Learning and Connecting, The Royal College of Physicians and Surgeons of Canada

Purpose/problem statement
To better reflect the lifelong learning journey of physicians, we propose a move towards the terminology Evolution of Competence. This term would better reflect the complexity of career development and the embedment of the philosophy of competence by design into that of continuing professional development.

Approach(es)
In certain jurisdictions, physicians are required to undergo regular testing to demonstrate their ongoing competence. Canada has opted to use a maintenance of certification (MOC) approach whereby yearly demonstration of participation in continuing professional development activities is assumed to confer ongoing competence in one’s original field of certification.

Findings
For many physicians, the MOC approach fails to acknowledge a significant and non-negligible evolution in scope and context of practice. Some cease clinical practice in their original field of certification and become more involved in medical administrative, leadership, advocacy or research work. Others narrow their field of expertise to a smaller subset of the work of their specialties while others still, evolve their work to include aspects that never existed when they were residents. These fellows are not maintaining their clinical skills and knowledge, they are evolving them to meet a need. While some may no longer be competent to practice in the full scope of their original field of certification, others contribute to a significant evolution of the practice itself. Nevertheless, using the MOC wording implies a status quo where everyone maintains their competence when in reality, the process is more dynamic and involves an evolution of competence during the continuum of one’s career.

Discussion
Done well, lifelong learning that integrates competence by design principles and CanMEDS through self-reflection is what leads to practice improvement and the evolution of the physician’s expertise over the course of their career. That evolution may thus naturally lead away from maintained competence in the initial field of certification to a new, tailored competence include a variety of new skills for individually evolving fellows. The physician whose practice has evolved away from their initial area of certification and who may no longer have the required competencies to practice in their certified specialty is no less a worthy and valuable contributor to the overall health of the Canadian population.

Barriers/facilitations
Nevertheless, the use of the “maintenance of certification” wording conveys the somewhat false impression that the physician has maintained their skills in their original area of certification, when in fact that may not be the case.

Impact/relevance to the advancement of the field of CME/CPD
As such, the term Evolution of Competence conveys the need to grow and change rather than to maintain. it shifts the philosophy of continuing professional development towards nurturing evolving competence and away from maintenance of certification.

45. Participation in Virtual Patient Simulation is Predictive of Documented Clinical Practice Change

Author(s)

Katie Lucero, MS, PhD, Vice President, Audience, Analytics, Outcomes, Medscape Education 

Don Moore, Jr., PhD, Professor of Medical Education and Administration, Emeritus, Vanderbilt University School of Medicine

Background/purpose/inquiry question
Continuing medical education (CME) with learning objectives which point toward outcomes that are competency- and performance-based come in many formats. We specifically were interested in examining the effect of a 1.0 credit CME/ABIM MOC patient simulation on documented real-world practice via medical and pharmacy claims data. The outcome of interest was selection of cardioprotective antihyperglycemic treatments for patients with type 2 diabetes (T2D). The simulation had two patient cases where primary care physicians (PCPs) and diabetologists and endocrinologists (diabetes specialists) made open-ended decisions about patient evaluation, diagnosis, treatment, and care management.

Theoretical framework(s)
Medical school training leverages simulation to prepare medical students for real world patient care, and research suggests simulation’s effectiveness. However, it is unknown whether current CME simulation for practicing HCPs offered in absence of quality improvement processes is associated with a real-world change in practice.

Methods
We utilized a post-hoc, pre-post, case-control design to examine the research question: Does participation in web-based patient simulation predict real world practice? The sample included 157 PCPs and diabetes specialists who participated in the simulation. They were case matched with non-participating HCPs on demographic and clinical practice characteristics. Logistic regression was conducted to examine whether use of cardioprotective antihyperglycemic treatments was predicted by participation in the simulation controlling for previous use of those treatments, specialty, and volume of patients with T2D.

Results/findings
Being in the CME simulation group predicted glucagon-like peptide-1 receptor agonist selection for patients with T2D after education (odds ratio 4.49; 95% CI 1.45-13.97; P=.001). The results suggest the effectiveness of voluntary web-based patient simulation on practicing PCPs and diabetes specialists.

Discussion
One of the challenges facing the field of CME/CPD is demonstrating value to stakeholders. One of the ways that the value of CME/CPD is described is clinician practice change. Past efforts have focused on commitment to change practice behavior or self-reported practice change measured by subjective responses to surveys after participation in CME activities. This study provides a more objective approach using claims data to show practice behavior change (selection of an appropriate treatment for T2D). In addition, the study used a novel approach, patient simulation in an online setting, to help clinicians learn about appropriate treatment of patients with T2D.

Limitations
Limitations of the research include it being central to one topic which is intensified treatment for patients with T2D and risk for cardiovascular events, localized to one virtual patient simulation, and potential unmeasured confounds that do not allow causal relationships to be identified but only casual inference.

Impact/relevance to the advancement of the field of CME/CPD
We suggest that further research will enhance the effectiveness of learning strategies using a combination of patient simulation and claims data to demonstrate the value of CME/CPD to stakeholders. Overall, studies like these communicate the power of CME/CPD to evoke measured change in practice.  

46. Recasting Assessment in Continuing Professional Development as a Person-Centered Activity

Author(s)

Helen Toews, MSc, Registered Dietitian, University of Toronto

Jacob Pearce, PhD, Principal Research Fellow, Australian Council for Educational Research 

Walter Tavares, PhD, Scientist, Assistant Professor, University of Toronto

Background/context
In this article we examine assessment as conceptualized and enacted in continuing professional development (CPD). Assessment is pervasive throughout the life of an individual health professional, serving many different purposes compounded by varied and unique contexts, each with their own drivers and consequences, usually casting the person as the object of assessment. Assessment is often assumed as an included part in CPD development conceptualization. Research on assessment in CPD is often focused on systems, utility, and quality instead of intentionally examining the link between assessment and the person.

Purpose/thesis statement(s)
We present an alternative view of assessment in CPD as person-centered, practice informed, situated and bound by capability, and enacted in social and material contexts.

Literature review/current perspective in the field
The assessment of practicing clinicians remains an important process and opportunity in CPD, but to date it has been given insufficient attention and is narrowly conceptualized. Assessment is pervasive throughout the life of an individual health professional, usually casting the person as the object of assessment. Assessment after entry-to-practice training becomes increasingly complicated and multifaceted, needing to account for the influence of, and response to, any combination of unique personal factors, and how the person relates to practice experiences, to understand what assessment can offer. CPD is an ‘umbrella’ term for a range of activities and practices, with the role and implications of assessment left implicit, often leaving the personal consequences of assessment unexplored.

Theoretical framework
With this lens of assessment as an inherently personal experience, we introduce the concept of subjectification, as described by educationalist Gert Biesta. (1,2) We propose that subjectification may be a fruitful way of examining assessment in a CPD context. (Reference: 1. Biesta GJJ, van Break. Beyond the Medical Model: Thinking Differently about Medical Education and Medical Education Research. Teaching and Learning in Medicine. 2020;32(4):449-456, 2. Biesta GJJ. World Centred Education: A View for the Present. 1st edition, Routledge; 2022)

Discussion
How does our perspective change when the health professional-as-a-person becomes the subject in assessment as opposed to the object of assessment? Biesta’s work invites us to consider a new way to conceptualize assessment in CPD, as a purpose-built opportunity to activate subjectification. This perspective opens the door to exploring assessment in CPD as a space for transformation of the health professional-as-a person, as a dialogue between the person enmeshed in their context and the external expert..The meaning and value assessment in CPD generates is very much shaped by person being assessed. Subjectification, as outlined by Biesta, problematises the enactment of assessment and calls our attention to what assessment does when the person is treated solely as an object of assessment. Subjectification calls us instead to consider how we engage health professionals-as-people in CPD, that assessment will always impact them as a person which influences what assessment can (or cannot) do for them as a result.

Impact/relevance to the advancement of the field of CME/CPD
The CPD context is different than the formal training context and is an inherently personal experience. This forces us to think differently about the role, structure, purpose, and consequences of assessment. Subjectification, as outlined by Biesta, calls our attention to recast how we think about assessment in CPD where the meaning and value of assessment is also very much shaped by those being assessed. Subjectification offers assessment as an opportunity to interrupt and refocus the health professional-as-a-person on their practice.

47. Strategies that Can Enhance Clinician Empathy and Compassion During Local and Global Disasters

Author(s)

Kathleen Snyder, Ph.D., Director, Magis Learning Partners

David Robinson, S.J., S.J., Ph.D., Educational Director, Magis Learning Partners

Purpose/problem statement
We are currently facing a world of erupting global climate disasters that are catastrophic in their impact on human life and well-being. These crises are occurring locally and globally and contributing to an even greater need for healthcare teams that can respond to the urgent call for help. The medical and psychological support needed for people impacted by the loss of life and the millions of people displaced is unprecedented for modern times. As the scope and impact of these events unfold, medical education, CME, and CPD will be required, to support medical professionals in developing compassionate and empathetic responses that can be integrated into their clinical practice.

Approach(es)
Recognizing that empathy and compassion are complex processes that can be developed as part of one’s practice is critical. They involve cognitive and affective strategies related to “genuine” human interaction,” that seeks to understand, respect, and support, regardless of the patient’s social or cultural background. Components include active listening, questioning, self reflection, critical thinking, decision-making, action, and engaged communication. This session provides participants with an overview of the processes and strategies involved in empathy and compassion, methods for integrating empathy and compassion in CME/CPD offerings and current research on empathy and compassion training.

Findings
Studies show that practitioner empathy and compassion can increase with education and training ( 2011, Riess H, Kelley JM, Bailey RW, Konowitz PM, Gray; 2013, Phillips M, Lorie A, Kelley J, Gray S, Riess H). Patient satisfaction has been demonstrated (2016, Boissy A, Windover AK, Bokar D, Karafa M, Neuendorf K, Frankel RM), in addition to improved patient experiences, health outcomes, reduced medical errors, and higher physician retention (2020, Moudatsou , Stavropoulou, Philalithis Koukouli), (2012, Riess H, Kelley JM, Bailey RW, Dunn EJ, Phillips M.)

Discussion
Empathic and compassionate clinicians are key to the future of healthcare and the response to local, national, and global communities in crisis. Empathy and compassion provide a basis for understanding needs and building relationships and the formation of communities that respond in crisis situations. Before these goals can be met, individuals and organizations need to recognize the importance of programs that include empathy training as a means for refreshing and expanding our health-care systems to better meet the challenging needs of today and the future.

Barriers/facilitations
Empathy and compassion lead to a better understanding of the needs of those suffering from the catastrophic events occurring today. CME/CPD professionals and medical educators can become part of the conversation that facilitates the development of these critical skills. Once acquired, these skills have the potential of contributing to new healthcare systems that are better prepared to address the new local and global healthcare crises that are emerging.

Impact/relevance to the advancement of the field of CME/CPD
Empathy and compassion are part of a holistic approach to medicine that integrates clinical skills with the cognitive and affective responses to patient needs. Promoting the development of these skills supports and advances the mission of CME/CPD in that it improves knowledge, awareness, competence, patient outcomes, and communities of support.

48. Virtual RSS: Health, Wellness & Professional Development Opportunities

Author

Annette M. Donawa, PhD, Associate Provost, Thomas Jefferson University

Purpose/problem statement
Regularly Scheduled Series (RSS) educational format is a widely common activity type among CME/CPD providers. According to the ACCME (2022) and Joint Accreditation (2021) annual reports combined, there were 27,638 RSS activities offered by accredited providers. For decades, RSS education was provided in-person. As a result of COVID, many providers pivoted to offering RSS education in virtual environments. Thomas Jefferson Office of Continuing Professional Development (OCPD) examined the potential impact of offering virtual RSS accredited education to learners regarding their health and wellness.

Approach(es)
OCPD expanded its research on RSS accredited education. Surveys were developed and distributed in 2021, 2022, and 2023. Learners were encouraged to reflect on barriers to attending RSS virtually and in-person. The most recent survey focused on the following questions: 1. Do you plan to attend your grand rounds in-person or virtual in FY24? 2. Does attending grand rounds virtually save time in your schedule? 3. Does attending grand rounds virtually support your personal budget? 4. Does attending grand rounds virtually improve your work/life balance? 5. What are some barriers for participating in your grand rounds virtually or in-person?

Findings
The responses from the learners were insightful (N=29). Question 1: 79% of learners intend to participate in RSS virtually, while 21% want to attend in-person. Question 2: 96% of learners agreed that attending RSS virtually saves time in their schedules while 4% disagreed. Question 3: 46% of learners believed that attending RSS virtually supported their personal budget and 54% did not. Question 4: 85% of learners agreed that attending RSS virtually improved their work/life balance while 15% disagreed. Question 5: Barriers to attending RSS virtually included experiencing technical issues and feeling distracted. Barriers to attending RSS in-person were time, location, and public transportation.

Discussion
CME/CPD providers should collect data on learner preferences in order to design targeted accredited educational programs. Conducting a deeper dive in understanding learner perceptions pertaining to attending RSS virtually versus in-person can benefit the healthcare team and the CME/CPD provider. Designing and delivering education to meet the needs of learners should be a priority to providers.

Barriers/facilitations
A potential barrier might include not having enough learner responses in this research study. Casting a wider net with the survey would increase the number of responses, which might impact the results of the data.

Impact/relevance to the advancement of the field of CME/CPD
Virtual and hybrid accredited education is here to stay is a part of our ‘new’ normal. Obtaining information on learner perceptions and the impact of learning in a virtual environment is critical to CME/CPD providers in designing meaningful education. Some learners shared that many presenters needed more coaching and skill building in teaching virtually, which could lead to targeted professional development opportunities.

49. Building Resilience and Well-being in Primary Care During and After the Covid-19 Pandemic: A Multiple Case-study in Continuing Professional Development

Author(s)

Francesca Luconi, PhD, Assistant Dean and Academic Associate, Office of Continuing Professional Development, McGill University

Rosario (Charo) Rodriguez, MD, PhD, Professor, McGill University

Mélanie Lavoie-Tremblay, PhD, Professeure Titulaire, Vice-doyenne, Université de Montréal

Jennifer Gutberg, MSc, PhD Candidate, Health Services Researcher and Consultant, University of Toronto

G. Ross Baker, PhD, Professor Emeritus/Emerita, University of Toronto

Tina Montreuil, PhD, Associate Professor, McGill University 

Jason Harley, PhD, Assistant Professor, McGill University

Peter Nugus, PhD, Assistant Professor, McGill University 

Catherine Briand, PhD, Professeure, Université du Québec à Trois-Rivières

Leonora Lalla, MD, Director of Continuing Professional Development, The College of Family Physicians of Canada 

Martin Desseilles, MD, Directeur du département, Université de Namur 

Brenna Lynn, PhD, Associate Dean, Continuing Professional Development, University of British-Columbia 

Suzan Schneeweiss, MD, MEd, FRCPC, Associate Dean, CPD, Temerty Faculty of Medicine, University of Toronto 

Özlem (Izzie) Barin, MSc, PhD Candidate, McGill University 

Sima Zahedi, MD, MSc Candidate, McGill University 

Béatrice Lauzon, PhD, Research Assistant, McGill University

Background/purpose/inquiry question
The role of primary healthcare providers (PCPs) is essential as it represents the frontline response to COVID-19 at the community level¹ and will contribute to prevention and containment of future crises. Stressful experiences during COVID-19 pandemic compromised PCPs’ well-being and quality of care^1-4 Continuing professional development (CPD) could play a key role in the pandemic and subsequent crisis by assisting PCPs in retooling and learning new skills and competencies⁵. COVID-19 revealed a substantial disparity between perceived psychosocial needs and available interventions (often at the individual level) for primary care providers. Research questions: 1. What are effective individual and team-level resources for building resilience among primary care providers (PCPs) in response to public health emergencies (e.g. Covid 19 pandemic)? 2. What are the barriers and enablers that limit or facilitate the implementation of resilience-enhancing resources across practice settings in primary care?

Theoretical framework(s) Integrated Knowledge Translation^,7 aims to close the research-practice gap to maximize the benefits of research within practice settings and policy. Informed by the Knowledge-To-Action framework⁸, this study is aligned with the knowledge synthesis, assessment of barriers and enablers, adaptation of knowledge to local contexts and dissemination of findings steps of the KTA Action Cycle. The transactional theory on stress and coping strategies guided data collection and interpretation of findings⁹.

Research Methods This multiple, descriptive case study¹⁰ included a mixed-methods literature review (1946-2021) and semi-structured interviews (2022-3, reported here). The main unit of analysis, i.e. the “case” is the impact of the C-19 on the mental health and well-being of community-based PCPs in Canada. Multiple case sampling (N =3 provinces) was selected to strengthen confidence, validity, and stability of findings. Criteria to select the provinces of Quebec and Ontario include: a) density of Francophone and Anglophone populations; b) concentration of physicians and nurses; b) hardest-hit provinces with reported C-19 cases;¹¹ c) concentration of academic centers and licensing regulatory bodies; d) feasibility and access; the PI and most of the research team are in these provinces which allowed leveraging cross-sectional networking to identify study participants and e) incidence of C-19 variants. The third province British Columbia (BC) was selected for its collaborative, community-based, team-based networks, the Patient Centered Medical Home model as well as its geographical position in Western Canada.

Participants: Snowball sampling and purposeful sampling strategies with maximum variation were used to recruit participants. Criteria for maximum variation included a) profession (i.e., family physicians, nurses/nurse practitioners (NPs), and psychologists); b) primary care settings  (i.e., clinics, hospital, emergency departments and long-term homes); c) type of practice (i.e., solo or interdisciplinary/interprofessional team(s) d) demographics (i.e., age, gender, years in practice, setting; country of origin, ethnicity) and e) experience with infectious diseases crisis. Participants were interviewed by videoconferencing in English or French. A questionnaire measured demographic variables, level of stress, and intention to quit. Member checks and triangulation of sources were used to increase the trustworthiness of the study. Data analysis: hybrid (deductive-inductive) thematic analysis and descriptive statistics (SPSS-29.0).

Results Questionnaire: Socio-demographics: 49 interviews (35% physicians, 30% (20% nurse 10% NPs), 31% psychologists) practicing in Quebec (39%%); Ontario (30%) and BC (30%). 65% practiced in urban areas and 19% in suburban areas; 73% in group practice; 69% interprofessional teams and 69% in interdisciplinary teams (often or very often). 20% were men, 78% were women, 2% preferred not to say. Age: 31% (50-59 years); 29% (40-49 years) and 20% (30-39 years). 31% more than 25 years of practice; 86% Canada as country of origin; and 76% Caucasian. 24% reported having been involved in managing the H1N1 crisis. Level of stress: 47% considered quitting their job (2020-2022) due to work overload (56%). The overall level of stress in 2021: 33% = stress level 5 or less; 67% = stress level of 6 or higher. Sources of stress: Work-related: 71% unclear/changing directives and 12% lost colleague or other person to C-19. Personal: 53% isolation and 45% having tested C-19 positive.

Interviews: (N = 4504 statements) Sources of stress: Systemic level: N = 298 (7%); work conditions = 147 (49%); changing directives = 50 (17%) and lack /limited Gov support = 47 (16%). Organizational level: N= 172 (4%) Lack or limited support from leadership; staffing; reassignment, top-down approach to deal with the pandemic. Individual level: N= 183 (4.1%): Uncertainty 67(37%) and family related stressors 45 (25%).

Resources: N = 318 (Organizational (i.e., offered at the ER, hospitals, teams) = 206 (65%); Systemic (i.e., offered by unions or government) = 87(27%); Individual level (i.e., formal tools whose use is self-initiated) = 25 (9%).

Enablers to access resources: N =132: Organizational: 59 (45%); Systemic: 54 (41%); Individual 19 (14%). Examples at the organizational level:  a) coordination/advertising (27%); b) leadership (20%); staffing (15%) and caring culture (15%).

Barriers to access resources: N = 219: Organizational: 79 (36%); Systemic: 100 (46%); Individual: 40 (18%). Stigmatization of mental health at the individual and systemic levels was reported.

CPD role to increase PDPs resilience: Offered regularly as accredited CPD activities; relevant by targeting PCPs needs; covering these topics: communication, work-life balance, psychosocial security; mindfulness; public health. Integrated with another topic to increase participation. Type of resources: webinars, podcasts, coaching, peer-to-peer support. Format: in-person/virtual. Venue: workplace. Promoted by local champions and leaders.

Discussion
Overall the findings of the interviews conducted with PCPs during C-19 confirm our mixed-methods literature review, including the need to combine individual and organizational-level interventions to support PCPs resilience and well-being during the C-19 pandemic. Organizational and systemic level factors are of central importance to support PCPs resilience. PCPs perceived that stigmatization of mental health is one of the barriers to access resources. This aligns with existing literature¹² The well-being and resilience of primary healthcare providers (PCPs) is essential as they represent the frontline response to C-19 and beyond. The C-19 has exacerbated pre-existing structural systemic gaps in the health system affecting PCPs well-being and resilience. Addressing these organizational and systemic gaps is crucial to PCP resilience and well-being. However, a paradigm shift from traditional CPD to competency-based, system-integrated CPD¹³  is needed to: a) demystify mental health at the individual, organizational and systemic levels; b) break silos within the healthcare system, and c) advocate for leadership support to change the healthcare culture across the educational continuum.

Limitations
Limited generalizability of findings; definition of primary care varies somewhat across the 3 provinces; variable terminology to identify CPD/CME resources and other resources to foster PCPs resilience. 

Impact/relevance to the advancement of the field of CME/CPD
Competency-based, system-integrated CPD interventions to foster professional and organizational resilience should contribute to demystifying mental health and advocate for leadership support and role modeling in healthcare culture. Advancing CPD research and scholarship around primary care provider resilience might support working practice environments.

References
1. Pollock A, Campbell P, Cheyne J, Cowie J, Davis B, McCallum J, McGill K, Elders A, Hagen S, McClurg D, Torrens C, Maxwell M. Interventions to support the resilience and mental health of frontline health and social care professionals during and after a disease outbreak, epidemic or pandemic: a mixed methods systematic review. Cochrane Database of Systematic Reviews 2020, Issue 11. Art. No.: CD013779. DOI: 10.1002/14651858.CD013779.

2. Lou NM, Montreuil T, Feldman LS, Fried GM, Lavoie-Tremblay M, Bhanji F, Kennedy H, Kaneva P, Drouin S, Harley JM. Evaluations of Healthcare Providers’ Perceived Support From Personal, Hospital, and System Resources: Implications for Well-Being and Management in Healthcare in Montreal, Quebec, During COVID-19. Eval Health Prof. 2021 Sep;44(3):319-322. doi: 10.1177/01632787211012742. Epub 2021 Apr 27. PMID: 33902348; PMCID: PMC8326888.

3. Naehrig D, Schokman A, Hughes JK, Epstein R, Hickie IB, Glozier N. Effect of interventions for the well-being, satisfaction and flourishing of general practitioners-a systematic review. BMJ Open. 2021 Aug 18;11(8):e046599. doi: 10.1136/bmjopen-2020-046599. PMID: 34408036; PMCID: PMC8375719. (More research is needed in this topic.

4. David E, DePierro JM, Marin DB, Sharma V, Charney DS, Katz CL. COVID-19 Pandemic Support Programs for Healthcare Workers and Implications for Occupational Mental Health: A Narrative Review. Psychiatr Q. 2022 Mar;93(1):227-247. doi: 10.1007/s11126-021-09952-5. Epub 2021 Oct 4.

5. Price DW, Campbell CM. Rapid retooling, acquiring new skills, and competencies in the pandemic era: implications and expectations for physician continuing professional development. J Contin Educ Health Prof. 2020 Spring;40(2):74-75. Doi: 10.1097/CEH.0000000000000297.

6. Graham ID, Tetroe J. How to translate health research knowledge into effective healthcare action. Healthc Q. 2007; 10:20–22.

7. Menear M, Grindrod K, Clouston K, Norton P, Légaré F. Advancing knowledge translation in primary care. Can Fam Physician. 2012; 58:623–627, e302

8. Graham, I.D., J. Logan, et al. (2006). Lost in knowledge translation: time for a map? Journal Contin Educ Health Prof 26(1):13-24.

9. Lazarus RS, Folkman S. Stress, appraisal, and coping. New York: Springer; 1984

10.  Yin RK. Case study research and applications: Design and methods. Sage publications; 2017.

11.  NY Times.com. https://www.nytimes.com/interactive/2020/world/canada/canada-coronavirus-cases.html

12.  David E, DePierro JM, Marin DB, Sharma V, Charney DS, Katz CL. COVID-19 Pandemic Support Programs for Healthcare Workers and Implications for Occupational Mental Health: A Narrative Review. Psychiatr Q. 2022 Mar;93(1):227-247. doi: 10.1007/s11126-021-09952-5. Epub 2021 Oct 4

13.  Price DW, Davis DA, Filerman GL. “Systems-Integrated CME”: The Implementation and Outcomes Imperative for Continuing Medical Education in the Learning Health Care Enterprise. NAM Perspect. 2021 Oct 4;2021:10.31478/202110a. doi: 10.31478/202110a.

50. Exploring the Current and Future States of AI and Emerging Digital Technologies in Specialty Medicine in Canada: Recommendations and Advancements from the Royal College Task Force on AI

Author(s)

Tanya Horsley, PhD, MBA, Associate Director, Research and Evaluation, Royal College of Physicians and Surgeons of Canada 

Mohsen Sheikh Hassani, BSc, MASc, PhD Student, Carleton University 

Richard Reznick, OC, MD, FRCSC, FACS, FRCSEd, FRCSI, FRCS, Immediate Past President, Royal College of Physicians and Surgeons of Canada

Purpose/problem statement
Recognizing the importance of advanced technology in health care, the Royal College Council commissioned a task force to help the medical profession in Canada prepare for the profound changes that artificial intelligence (AI) and emerging digital technologies will bring to residency training and delivery of care. A 14-member task force was convened to conduct an extensive review into the current and future states of AI & emerging digital technologies and to provide recommendations about how the Royal College could meet the challenges and opportunities these technologies present. These recommendations were tabled in 2019. Herein we present the data from an all-Fellow survey and organizational advancements since the delivery of the report.

Approach(es)
The task force consulted widely with key stakeholders, reviewed the current literature, surveyed Fellows, and Resident Affiliates, formally interviewed over 20 authorities in the field, and formally engaged a group of Fellows of the Royal College with expertise in AI and emerging digital technologies. Specifically, a survey was iteratively developed and distributed electronically to all 50,835 Fellows. The survey was available in English and French, and consisted of nine Likert scale questions, one drag-and-drop ranking question and two optional open-ended questions exploring the perceptions of AI both generally and associated with the participant’s specialty. Analysis of responses was focused on educational and institutional gaps and learning opportunities.

Findings
Quantitative and open-ended responses from 4, 497 respondents were analyzed using a content analysis approach independently by two project staff. This approach seeks to systematically read text and assigns identifiers to information that may be perceived as important or clustering around particular themes. Survey respondents indicated a general lack of exposure to AI both in practice and in training (specifically resident responses). While both Fellows and Resident Affiliates appear keen to engage as partners early in the innovation process (46.1% desire to be early adopters), a barrier is a lack of understanding of AI generally (44.8% self-assessed their familiarity with AI as either neutral, somewhat-, or not familiar at all). Respondents were asked to prioritize areas of focus for educational content as part of their Maintenance of Certification; general AI-related educational content, followed by ethics & legal considerations.

Discussion
There is a critical disconnect between survey respondents’ level of understanding of AI & emerging digital technologies and respondents’ desires to engage as early adopters and innovators of AI. This gap may be impeding engagement of learners and Fellows across Canada. These data have informed strategic advancements with a recognition that digital health literacies were viewed as a critical educational gap; these should be integrated into the CanMEDS framework. To build capacity, the RC is advancing a new Area of Focused Competence in Canada.

Barriers/facilitations
NA

Impact/relevance to the advancement of the field of CME/CPD
This is the first National survey of specialists in Canada to explore the level of understanding of AI and what residents and Fellows would prioritize for learning as part of their continuing professional development. The findings are drawn from a population in Canada but may be generalizable to North America.