Whose Needs are you Assessing Anyway: Why the Unit of Analysis Matters for CPD Needs Assessments?

By Helen Mawdsley, EdD and Eleftherios K. Soleas, PhD

We know that drawing on multiple data sources for needs assessment is a practical way to overcome the limitations that physicians have in their ability to accurately self-assess their learning needs (Davis, Mazmanian, Fordis, Van Harrison, Thorpe, & Perrior, 2006). We agree, and would like to maintain that drawing on multiple data sources for needs assessment is indeed prudent; however, we would like to discuss the point of view of how the unit of analysis within CPD needs assessment needs some attention as well. Specifically, what if we told you that a perfectly valid target of a needs assessment is establishing the needs of the community, rather than the clinician?

In needs assessments, who you ask very much determines what you will find. Out of respect for the time of readers, we’ll put this question frankly: if the goal of healthcare is better community health, shouldn’t the most common or at least co-common audience of needs assessments be the community? If we accept the premise of this first question, the natural next is what does a needs assessment look like that works for community needs? The simple answer is non-existent, but the more complex answer necessitates a rebalancing of perceived (usually healthcare provider derived) and unperceived (typically community and systems-based) needs dependent on higher-order evaluations and outcomes data. So what stands in the way of doing that? Here are a few likely suspects for your consideration:

• Access to data. Higher-order evaluation data does not grow on trees, but it can be acquired although access typically requires effort, resources and partnerships.
• Community ‘buy-in’ data. These data are most frequently systems-level data and come from a combination of large-scale studies, some of which are published and available in the literature.
• Ethical clearance and ability to de-identify data. Systems-level data tend to be highly protected for excellent reason, therefore strong ethical protocols and partnerships are necessary to access these data.
• Availability of skillsets. The statisticians who perform the necessary data analysis for these needs assessments don’t grow on trees and no matter how much they talk about ‘z-scores’ or ‘normality’ we can’t escape the need for them.
• Evaluating at the community-level vs clinical-level. While it may make it more challenging to directly attribute changes to CPD, it’s much harder to observe these higher-order outcomes, and to understand if the CPD intervention is working or not.

A CPD needs assessment where the unit of analysis is the clinician, relies on clinician self-assessment.

Epstein, Siegel, & Silberman (2018) stated that clinician self-assessment is a reflective process in which the clinician compares their performance to a standard. With an absence of standards, or standards created in different contexts than in which one is practicing, self-assessment for clinicians is challenging. Doable, but challenging. Typically, this style of needs assessment looks like surveys sent to clinicians, asking them about their learning needs, and maybe even the learning needs of their colleagues. These surveys often ask about areas they would like to expand into, areas which they feel they struggle in, areas which they may feel they need updating on (such as new practice guidelines or new pharmaceuticals), and even challenging patient cases. Some surveys may even go so far as to ask about the patient population which they serve; however, this is all from the point of view of the clinician. If we were to extend to the point of view of the community – what would this look like? Furthermore, if we did extend our needs assessment to include the view of the community, would this enable CPD programs to evaluate effectiveness at the community level? Undoubtedly, this opens up another area for discussion on how to access data longitudinally and how to attribute CPD educational interventions to changes at the community level. (We look forward to discussing this more in upcoming CE News issues.)

When conducting CPD needs assessments with the clinician point-of-view as the unit of analysis, we are probably gathering robust data on perceived clinician needs. However, this may or may not meet the needs of patients. We know that people avoid seeking medical care, and that this number is as high as one-third of the population in the US alone (Taber, Leyva, & Persoskie, 2015). A recent study in Canada shows that racism still serves as a major barrier in accessing health and social services (Monchalin, Smylie, & Nowgesic, 2020). Further, there are system barriers which challenge access for new Canadians (Salamo, Mason, Salma, Yohani, Amin, Okeke-Ihejirinka, & Ladha, 2020), and for those living in material and social deprivation (Loignon, Hudon, Goulet, Boyer, De Laat, Fournier, Grabovschi, & Bush, 2015), among many other stories of people who are unable to access health care, even when they live in a country with a universal healthcare system. We think it is important to consider the actual recipients of healthcare – currently and in the hopeful future, as access is expanded. In order to capture broader community needs the CPD needs assessment process should consider the community as a unit of analysis, in addition to continuing with the clinician as a unit of analysis.

So where to now? Do we advocate for removing the clinician point of view from needs assessment? Absolutely not. We need to strike a better balance of assessing the needs of the clinicians and the needs of those that clinicians serve – the actual recipients of healthcare.

References

1. Davis, D. A., Mazmanian, P. E., Fordis, M., Van Harrison, R. T. K. E., Thorpe, K. E., & Perrier, L. (2006). Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. Jama, 296(9), 1094-1102.
2. Epstein, R. M., Siegel, D. J., & Silberman, J. (2008). Self-monitoring in clinical practice: a challenge for medical educators. The Journal of continuing education in the health professions, 28(1), 5–13. https://doi.org/10.1002/chp.149
3. Loignon, C., Hudon, C., Goulet, É., Boyer, S., De Laat, M., Fournier, N., Grabovschi, C., & Bush, P. (2015). Perceived barriers to healthcare for persons living in poverty in Quebec, Canada: the EQUIhealThY project. International journal for equity in health, 14(1), 1-11.https://doi.org/10.3102/10769986002003171
4. Monchalin, R., Smylie, J., & Nowgesic, E. (2020). “I guess I shouldn’t come back here”: Racism and discrimination as a barrier to accessing health and social services for urban Métis women in Toronto, Canada. Journal of racial and ethnic health disparities, 7(2), 251-261.
5. Salami, B., Mason, A., Salma, J., Yohani, S., Amin, M., Okeke-Ihejirika, P., & Ladha, T. (2020). Access to healthcare for immigrant children in Canada. International journal of environmental research and public health, 17(9), 3320.
6. Taber, J. M., Leyva, B., & Persoskie, A. (2015). Why do people avoid medical care? A qualitative study using national data. Journal of general internal medicine, 30(3), 290-297.