Educational Research

Program Evaluation

The goal of our program evaluations is to determine the effectiveness of the PBSG Learning program and the level of satisfaction our members have with the program.

PBSG Member and Facilitator Surveys are important components of the program evaluation process. The surveys offer program participants the opportunity to provide feedback on a range of issues and make suggestions for further enhancement of the program.

Members are also offered the opportunity to evaluate each PBSG module. This feedback is invaluable to the module development team and helps create modules that better meet the learning needs of members.

In addition to member feedback, we regularly evaluate the program internally at all levels of our organization. Assessments from all stakeholders are used to determine whether our educational goals are being met and whether our program is evolving to meet emerging needs.

Past efforts to improve the program are reflected in increasing member satisfaction over time: 

Membership Survey Results: 

The overall PBSG learning experience; good to excellent:

          2013          95.2%

          2010          90.8%

          2005          89.7%

The quality of group interaction; good to excellent:

          2013         97.7%

          2010         92.9%

          2005         89.4%

The overall quality of the modules; good to      excellent:

        2013         88.3%

        2010         81.3%

        2005         80.5%

Research

The goal of the FMPE Research program is to:

“Clarify the processes and components of the PBSG learning program that enhance physician learning and practice change”

The first funded research project undertaken by FMPE was the Better Prescribing Project (1998-2000) designed to evaluate the effectiveness of the PBSG program. This study showed that physicians who use the PBSG process are more likely to change their prescribing behavior (Herbert 2004; Wakefield 2003; Wakefield 2004).

In order to facilitate a greater focus on research, a formal research program was established in 2009. The program has several ongoing projects, conducted by FMPE faculty and also in collaboration with a number of renowned medical education researchers. Topics of research include:

  • practice reflections
  • commitment-to-change
  • communities of learners
  • self-assessment
  • knowledge translation
  • practice implementation

These research projects would not be possible without the full support and voluntary participation of PBSG facilitators and members.

The following agencies have provided funding for peer reviewed projects:

  • Health Canada
  • Society for Academic Continuing Medical Education (SACME)
  • Medical Council of Canada (MCC)
  • Canadian Institutes of Health Research (CIHR) 

Publications

Armson, H, Elmslie, T, Roder, S, Wakefield, J. Encouraging Reflection and Change in Clinical Practice: Evolution of a Tool. JCEHP 17 SEP 2015;35(3); 220-231. PMID: 26378428

Armson, H, Elmslie, T, Roder, S, Wakefield, J. Is the Cognitive Complexity of Commitment-to-Change Statements Associated With Change in Clinical Practice? An Application of Bloom's Taxonomy. JCEHP 17 SEP 2015;35(3); 166-175. PMID: 26378422

Armson H, Wakefield J. Expanding the horizons of practice-based small group learning: what are we learning? Educ Prim Care 2013; 24(3): 153-155. PMID: 23676868

Armson H, MacVicar R. (2013) The Thistle and the Maple Leaf: Practice-Based Small-Group Learning in Canada and Scotland. In: Clinical Uncertainty in Primary Care. Eds. Sommers, LS, Launer J. Springer New York. Chapter 6: 117-146.

MacVicar R, Guthrie V, O’Rourke J, Sneddon A. Supporting educational supervisor development at the interface: evaluation of a pilot PBSGL for faculty development. Educ Prim Care 2013; 24(3): 178-84. PMID: 23676873

Armson H, Kinzie, S, Hawes D, Roder S, Wakefield J, Elmslie T. Translating Learning into Practice: Lessons from the Practice-Based Small-Group Learning Program. Can Fam Physician 2007; 53: 1477-1485. PMID: 17872876

Hesselgreaves H, MacVicar R. Practice-based small group learning in GP specialty training. Educ Prim Care; 2012; 23(1): 27-33. PMID: 22306142

Cunningham D, McCallister P, MacVicar R. Practice-based small group learning: what are the motivations to become and continue as a facilitator? A qualitative study. Qual Prim Care 2011; 19(1): 5-12. PMID: 21703107

Armson H, Kinzie S. (2010) Evidence-based Learning and Practice: A Guide for Small Group Facilitators. Wakefield, JW. Editor. Hamilton, Ontario: The Foundation for Medical Practice Education. p. 1-14.

Sargeant J, Armson H, Chesluk B, Dornan T, Eva K, Holmboe E, Lockyer J, Loney E, Mann K, van der Vleuten C. The Processes and Dimensions of Informed Self-Assessment: A Conceptual Model. Acad Med 2010; 85(7); 1212-1220. PMID: 20375832

Roder S, Armson H, Kinzie S, Shaw E, Elmslie T, Wakefield J. (2010) Learning from practice – using multiple types of program evaluations for continuous improvement of the Practice-Based Small Group Learning Program. Family Medicine Forum Oct 2010.

Walsh AJ, Armson HA, Wakefield JG., Leadbetter W, Roder, S. Using a Practice-Based Small Group Learning Approach to Enhance Effective Feedback Skills for Teachers. Teach Learn Med 2009; 21:45-51. PMID: 19130386

Overton GK., McCalister P, Kelly D, MacVicar R. Practice-based Small Group Learning: How health professionals view their intention to change and the process of implementing change in practice. Med Teach 2009; 31:e514-e520. PMID: 19909029

Overton GK, MacVicar R. (2008) Requesting a Commitment to Change: Conditions that produce behavioral or attitudinal commitment. J Contin Educ Health Prof 2008; 28(2): 60-66. PMID: 18521882

MacVicar R, Cunningham D, Cassidy J, McCalister P, O’Rouke J, Kelly D. Applying evidence into practice through small group learning: A Scottish pilot of a Canadian programme. Educ Prim Care 2006; 17(5): 465-472.

Herbert CP, Wright JM, Maclure M, Wakefield J, Dormuth C, Brett-MacLean P, Legare, J, Premi J. Better Prescribing Project: a randomized controlled trial of the impact of case-based educational modules and personal prescribing feedback on prescribing for hypertension in primary care. Family Practice 2004; 21 (5): 575-581. PMID: 15367481

Wakefield JG. Commitment to Change: Exploring its role in changing physician behavior through continuing education. J Contin Educ Health Prof 2004; 24:197-204. PMID: 15709559

Wakefield J, Herbert CP, Maclure M, Dormuth C, Wright JM, Legare, J, Brett-MacLean P, Premi J. Commitment to change statements can predict actual change in practice. J Contin Educ Health Prof 2003; 23: 81-93. PMID: 12866327

MacVicar R. Canada’s practice based small group learning programme: An innovative approach to continuing professional development. Educ Prim Care 2003; 14: 431-439.