Dr. Stefanie Roder is a trained Neurophysiologist with more than 25 years of research experience in both neuroscience and continuing medical education. Since 2005, Stefanie has been collaborating with Dr. Heather Armson and other medical educators on various research projects on physician small group learning and implementation of knowledge into practice.
Q: Who were the authors of this study?
The authors of this study are:
Dr. Heather Armson, FMPE Executive Director and Research Director
Dr. Kathleen Moncrieff, FMPE Assistant Research Director
Meghan Lofft, Former Research Coordinator for module development at FMPE
Dr. Stefanie Roder, Coordinator of the research program at FMPE
Funding for this project was provided by an internal research fund through the Foundation for Medical Practice Education, McMaster University in Hamilton, Ontario, Canada
Q: Why did the study choose an ethnographic approach?
Ethnography is a qualitative research methodology that predominantly collects observational data in natural settings to understand how people interact with each other and their environment. The benefit of ethnography is that the research takes place “in the field” without set conditions created by the researcher.
In our study, we wanted to gain insights into how physicians interpret new information, assess its applicability within the context of their own practices and better understand what contributes to decisions to change practice during small group learning sessions. We felt direct observation of small group sessions would provide more rich data to help answer our research questions as compared to more limited data collected from questionnaires and interviews alone.
Q: How did observational dimensions and thematic content analysis help understand physician discussions?
We used the observational dimensions of the ethnographic method to guide the initial analysis of the field notes to understand the environmental context of the small group discussions that were observed. There are nine observational dimensions – space, actors, activities, objects, acts, events, time, goals, and feelings. We tabulated these for all nine observed small group learning sessions to get a good understanding of who was present at the sessions and what was happening during the sessions. It provided us with an overview of the structure of the physicians group discussions and helped us make comparisons across all observed small group sessions.
We also used the thematic content analysis to understand the content of the small group discussions captured in the field notes. Two approaches were used: 1. the conventional approach derived themes directly from field notes of the conversation pieces made by physicians during the sessions. The emerging themes during the coding process were used to identify elements of the change talk, and 2. the directed approach coded conversations pieces specifically to our research questions of this study.
Q: How did facilitators lead discussions by focusing on practice gaps?
Facilitators usually started the small group sessions by asking about “thoughts on the module” or “anything surprising in the module”. Members would bring forward their reflections on the module topic which at times would include identifying perceived gaps in practice. In some sessions, it was observed that the facilitator would directly ask “Do you have any gaps in practice?” or “any gaps identified?”.
The facilitator also fills out the practice reflection tool (PRT) at the end of the small group sessions, and together with group members completes the section on the PRT which asks about “gap(s) in our current practice”. Members will state the gaps in practice that they talked about during the small group session.
Q: How did facilitators contribute to identifying practice gaps?
The facilitators are peer members of the group as well. Thus, facilitators would not only stimulate group members to think about practice gaps but would also share their own personal practice gaps related to the module topic with their group members.
Q: Can you share examples of physicians sharing approaches to clinical cases?
During the case discussions physicians shared their approaches to the module cases. A variety of themes were identified from the field notes; these could be around history taking, physical exams, investigations, treatment, and others. Specific examples are: “Basic blood work and try to convince them to stop smoking” (Prenatal Screening Module); “ask where it hurts before examining; compare sitting, standing, walking” (Acute Sports Injury Module) and “stop nasal decongestants” (Cough in Adults).
Q: How did physicians make sense of or interpret new information during discussions?
During the small group sessions, physicians interpreted new information by:
- Reflecting on the new information and identifying gaps in knowledge (“wouldn’t have thought that…” / “Never heard of …” / “did not know this”);
- Asking questions and clarify understanding (“way I understand is…” / “do you know why…”);
- Assessing evidence (“isn’t evidence better for…” / “summary – not great evidence”);
- Expressing need for more information if it does not make sense (discussing gaps in modules or say what “should have been included”).
Q: How did physicians determine the usefulness of information, as well as assess applicability of information to clinical practice?
During the small group sessions, physicians assess applicability of new information to clinical practice by:
- Asking questions (“would it overwhelm the patient?” / “I wonder if ___ can be used instead of ___”);
- Testing out algorithms /practice tools (“let’s go to flowchart and look at inclusion and exclusion criteria”);
- Reflecting on their comfort level (“I would feel [un] comfortable doing this” / “I hesitate adding on medications”);
- Sharing patient stories /practice implementation experiences (“My patient had a good response with steroids … my patient was the same”).
Q: Can you describe the role of evidence review and knowledge consolidation in decision-making?
The level of evidence for new recommendations for clinical practice is important to physicians in making any decisions for changes in practice – “it is the discussion around evidence and experience that makes me decide” – if the evidence is low (“this was not evidence based on expert opinion”) the physician will not rush to make a change to practice. But evidence alone may not be enough. Even if evidence is strong, new recommendations may still not be implemented – “sometimes the evidence is very strong and is challenging the way we practice, then we discuss whether or not this potential change can be rolled out in our environment”.
Knowledge consolidation is a process by which knowledge from various sources (including level of evidence) are merged/ linked together to provide a better understanding of the topic being studied and help make appropriate decisions based on this combined knowledge.
Knowledge consolidation in this study became evident when parts of a case discussion came to an end and the facilitator summarized a consensus and/or specifically asked for decisions for practice. Examples were: “Examining above and below, look for compartment syndrome – good reminder to be thoughtful and systematic” (Acute Sports Injuries Module) and “let’s summarize our decision. Would we treat for GERD?” (Cough in Adults Module).
In the absence of knowledge consolidation, physicians might not make a decision to change practice, but instead look for additional information after the learning session to help decide whether a practice change should or can be made.
Q: Provide examples of how shared practice experiences influenced decisions?
Sharing of practice experience is a big component that draws physicians to small group learning. Small group learning not only helps physicians benchmark themselves to new recommendations provided within modules but practice experience from other physicians within the group.
One physician in the study said: “Sometimes people have experience from their own practice that you haven’t seen that is helpful in terms of change”. When group members share their approaches to clinical practice, others can reflect and compare on what they do in practice themselves; this in turn may help physicians become aware of the gaps they might have or what approach(es) might work better for specific clinical challenge(s).
Once a physician is aware of their own practice gap(s)/ challenge(s) they can decide what to do about it. Again, if a member of the group is already following best practices, they can share and provide others with what works and does not work within the context of their own practice. They might provide additional information and resources needed to decide if a practice change is warranted and provide suggestions to help overcome barriers to implementation of new best practices.
Q: How did integrating practice experiences validate guideline recommendations?
A physician that was interviewed in this study did say, “it’s the discussion around evidence and experience that makes [them] decide”. It points to the fact that as physicians talk about their practice experiences, they consider new guideline recommendations by sharing their experiences of what works and what does not work in the context of their own practice.
The new practice recommendations may be great, but shared practice experiences may tell a different story. One physician pointed out that, “sometimes the evidence is very strong and is challenging the way we practice, then we discuss whether or not this potential change can be rolled out in our environment and becomes more operational or more practical”.
If practice experiences from others do support the new guideline recommendations, then a physician will be convinced to try to implement new guideline recommendations. However, if it does not fit within the context of their own practice then a decision to implement a change will not be supported or delayed pending further investigations into evidence around the new guideline recommendations.
Q: How did practice reflection documents contribute to understanding decision-making?
The practice reflection documents are completed at the end of the learning sessions, the facilitator together with group members document not only what information was useful and what practice gaps they have related to the clinical topic discussed, but also what changes they plan to implement into their practice.
Analysis of the practice reflection documents gave indications of which information discussed during the session was important to the physician in making decisions for practice such as “evidence for … treatment options” or “colleagues’ personal experiences and practical tools and tips”. Any documented practice gaps of the practice reflection documents showed awareness of what could be changed and often was also linked to the documented planned practice changes. The practice reflection documents also provided additional insights into the decision making when considering the barriers and enablers to practice implementation.
It should be mentioned that we do have a publication that describes the evolution of the practice reflection tool that was used in this study. Anyone interested is invited to read Armson et al 2015 Encouraging Reflection and Change in Clinical Practice: Evolution of a Tool. J. Cont. Educ. Health Professions 35(3):220-31. In this paper we provide evidence from literature, membership surveys and facilitator’s interviews on how the practice reflection tool promotes reflection on current practice and helps with the articulation of planned practice changes.
Q: Can you briefly explain the observed overlaps between documented reflections and field notes?
Field notes taken at the end of the session when the facilitator summarized the outcomes of the small group learning session were compared to the documented reflections submitted to the learning program after the session. This was done to determine the extent of overlap of the two data sources.
Similarities in documented information between the two data sources enhanced credibility of the data collected and any differences provided additional concepts for decision making. There was a significant overlap between the two data sources, confirming credibility of data collected, in particular to which information was important, practice gaps that were identified and which decisions were made for practice. Information on the PRT also supported that evidence, practice tips and experiences are important contributors to practice change decisions.
Q: How does the ‘change talk’ framework help understand bridging gaps between current and best practices?
The “change talk” framework was created within the context of the small group practice-based learning program to help conceptualize how physicians interpret information and assess application of new knowledge into practice during a learning session.
During the small group sessions, physicians talk through clinical cases provided within the educational material (module). They discuss feasible approaches to the hypothetical cases considering evidence-based information provided in the module in addition to their own clinical knowledge and practice experiences. They try to make sense of the new information by asking questions and sharing knowledge.
They determine what information is useful and whether it applies to clinical practice. They reflect on their own experiences and benchmark themselves to best practices recommended in the module and practice experiences shared by group members.
Only when evidence has been considered, new knowledge has been consolidated and feasibility of implementing practice change have been determined are physicians ready to make decisions for practice change (i.e. bridge the gap between current and best practice).
Q: What key elements of the ‘change talk’ framework were derived from the study?
The key elements of the “change talk” framework within the context of small group learning we identified in this study are knowledge sharing, knowledge clarification, knowledge application and knowledge consolidation. These elements will help with understanding evidence for best practice, practice reflections, benchmarking and sharing of tacit knowledge to make appropriate decisions for change.
Q: What are the broader implications of the study for the medical community?
Our study provides detailed insights into how physicians discuss clinical topics with the purpose of understanding and implementing new knowledge into practice. Small group discussions with colleagues can help overcome individual uncertainties in making appropriate decisions regarding diagnosis and treatment for optimal patient care.
The conceptual framework for change talk identifies conversational elements that occur during small group learning that can lead to decisions for practice change. Medical educators need to be aware of these conversational elements to promote educational strategies using interactive approaches to learning that include understanding evidence for best practice, practice reflection, benchmarking and sharing of tacit knowledge to promote appropriate decisions to maintain practice competence.
Q: Do you think the ‘change talk’ framework could apply to other medical specialties?
The conceptual framework for change talk needs to be verified in other educational settings using small groups as strategy for learning and practice implementation. We believe that it does apply to other medical specialties using small group learning as an educational strategy to understand gaps in practice and discuss strategies to implement new best practices.
As a matter of fact, our paper on change talk received a commentary in the same journal (Looman et al 2023 “No doctor is an island” Medical Education 57(11) 1-3) with the authors commenting that the change talk framework could be considered for interprofessional (mix of physicians and specialist physicians) communities of practice to overcome challenges they may encounter in practice.