Printed from: http://www.stfm.org/NewsJournals/FamilyMedicine/PastIssues/ArticleView

Expectations of the Competencies of Entering Family Medicine Residents: Do Members of the Family Agree? A CERA Study

Sarah J. Swofford, MD, MSPH; Amanda L. Allmon, MD; Christopher P. Morley, PhD; Robin L. Kruse, PhD; Elizabeth Garrett, MD, MSPH

Background and Objectives: Little information is available on what competencies family medicine program directors expect of incoming residents and whether those incoming residents can deliver on such expectations. Our aim was to determine whether there is a common set of skills that both family medicine clerkship directors and program directors agree are important for entering residents to possess and how often these are being demonstrated.

Methods: This study is an analysis of data obtained in the 2014 CERA Family Medicine Clerkship Director and Program Director surveys. Clerkship directors and program directors were provided with a list of skill areas and professional characteristics and asked to rate how important it is for medical school graduates to demonstrate them as well as how many medical school graduates consistently demonstrate each skill or characteristic.

Results: Both groups placed highest importance on honesty, professionalism, and communication skills among incoming residents. Clerkship directors placed higher importance on several skill areas, with program directors emphasizing professional characteristics. This division was also seen in the opinions of the two groups for how well entering residents demonstrated each area.

Conclusions: Clerkship directors identify several skill sets as being more important for graduating medical students to possess compared with program directors. Program directors seem to value more personal characteristics as important for entering residents to possess rather than skills. Future implications may include a focus from family medicine clerkships on skills that are valued by program directors but not rated as commonly seen among incoming residents.

(Fam Med 2016;48(8):613-17.)

Family medicine clerkship directors have a broad set of curriculum goals and objectives for their clerkships whose lengths can vary from 4–8 week blocks to longitudinal scheduling across an entire year. Many different groups within medical education and the specialty have defined goals and objectives for medical student education. Medical educators are responsible for producing graduates with basic competencies in a wide range of areas with the national Family Medicine Clerkship Curriculum,1 focusing on patient care; knowledge and skills for acute, chronic, and preventive care; office visits; and the principles and roles of family medicine in our health care systems.

Competencies defined by the Accreditation Council for Graduate Medical Education (ACGME) have been incorporated into residency training since 2007, and medical schools are now examining these six competency areas to determine which are most relevant to undergraduate medical education.2 Within medical schools, family medicine clerkship directors and faculty regularly examine their curricula to make sure they meet the goals and objectives of medical schools, the Liaison Committee for Medical Education, and specialty standards. However, anecdotally, there has been increasing concern expressed at national meetings by some family medicine residency program directors that entering residents are less well-prepared than past medical school graduates. Often the implication is that medical schools are failing in their responsibilities to provide a well-rounded, general education to their students. There is also concern expressed over the wide variability in the apparent preparedness of US graduates entering family medicine residency programs. These concerns mirror those reported across specialties for a number of years.3,4

A number of family medicine researchers have described predictors of residency success, as well as attempted to characterize program director expectations for incoming residents. For example, Paolo et al demonstrated that higher medical student third-year GPAs and USMLE step 1 and step 2 scores correlate with more favorable residency program director ratings.5 In terms of new family medicine resident performance, Langdale et al examined what entry-level procedural skills and competencies residency directors agreed that medical students should possess but not whether students were actually meeting those expectations.3 Murdoch et al showed that experience and comfort level regarding basic procedures in family medicine varies among incoming residents,6 and Dickson et al showed that at least 66% of family medicine residency directors expected interns to enter residency able to independently perform 15 procedures but that more than 80% of new interns self-reported being able to independently perform only five of these 15 procedures.7 Because of this variability and uncertainty, many residency programs include early PGY-1 assessments to establish baseline clinical skills.8

Despite these observations, the extent of agreement between family medicine program directors and clerkship directors regarding competencies of incoming residents has not previously been reported, and there has often been a perceived mismatch between priorities as viewed by these two groups.9 Having a better understanding of family medicine program directors’ perceptions of the strengths and weaknesses of their incoming residents, and contrasting those perceptions with those held by family medicine clerkship directors about their own students, would have wide-ranging implications. To gain insight into the contrasting perceptions between family medicine program directors and clerkship directors, we sought to engage in a two-part inquiry asking both the producers (clerkship directors) and the consumers (program directors) where they believe gaps in preparedness currently exist for US medical school graduates. The results of this study are intended to help guide a broader, more substantive discussion across the “family” of family medicine educators to help reach consensus for how best to prepare medical students for residency training in an era that is constantly changing regarding graduate education and health care delivery.

This study builds on a pilot project that identified areas of overlap and areas of discordance that exist between the expectations of family medicine clerkship directors and program directors regarding entering residents’ skills. Our aim was to determine whether there is a common set of skills that both family medicine clerkship directors and program directors agree are most important for entering residents to possess. We also sought to identify how often clerkship directors and program directors felt those skills or characteristics were demonstrated by graduating medical students or entering interns.

 

Methods

 
 

The Council of Academic Family Medicine (CAFM) Educational Research Alliance (CERA) conducts annual surveys of clerkship directors and program directors.10 CERA is a joint initiative of all four major US academic family medicine organizations (Society of Teachers of Family Medicine, North American Primary Care Research Group, Association of Departments of Family Medicine, and Association of Family Medicine Residency Directors). This study is an analysis of data obtained as part of the 2014 CERA Family Medicine Clerkship Director Survey and the CERA Program Director Survey. The study was approved by the American Academy of Family Physicians Institutional Review Board.

The CERA Family Medicine Clerkship Director Survey is distributed annually to the institutional representatives of qualifying medical schools. The institutional representative is the clerkship director at the main campus of the school or his/her designate. The survey was conducted between September 1, 2014, and October 20, 2014. The CERA Program Director Survey is distributed annually to the family medicine program directors who are Council of Academic Family Medicine (CAFM) members. The survey was conducted between January 29, 2015, and March 12, 2015. Survey items were developed for use on both surveys by the authors. CERA provided us with de-identified results for analysis.

Respondents were asked to provide basic information about their schools and clerkship or residency programs. In addition, they were presented with a list of skill areas and professional characteristics identified from a previous small-scale qualitative study as important for medical school graduates entering residency training.9 Respondents were asked to rate how important each was for medical school graduates to demonstrate as well as how many graduates consistently demonstrate the skill or characteristic. To reduce survey size, program directors rated half of each list of skills or characteristics based on their selection from a list of numbers.

Clerkship directors and program directors were provided with a list of skill areas and professional characteristics and asked to rate how important it is for medical school graduates to demonstrate each skill or characteristic. Responses were assigned a Likert scale from 1 to 5, with 1=good but not important, 2=not very important, 3= important, 4=very important, and 5=absolutely required.

Respondents were also asked to rate how many medical school graduates consistently demonstrate each skill area or professional characteristic. Responses were assigned a Likert scale from 1 to 5, with 1=zero or almost zero, 2=few, 3=some, 4=many, and 5=all or almost all students.

Survey responses were analyzed using SAS for Windows 9.4 (SAS Institute, Inc., Cary, NC). Simple frequencies were used to describe clerkship and residency programs. Responses of clerkship and program directors were compared with the t test. A P value of .05 or less was used to indicate statistical significance. A sub-analysis of program director responses using ANOVA was performed by residency type (university versus community-based residency) and community size.

 

Results

 
 

The clerkship director surveys were distributed to 121 US and 16 Canadian clerkship directors. The overall response rate was 91%, with 124 out of 137 clerkship directors responding. The program director surveys were distributed to 452 family medicine program directors. There were 274 responses, for an overall response rate of 61%. One program director survey that was entirely blank was dropped from the analysis.

The characteristics of the participating programs are summarized in Table 1. Just under half of clerkship directors were women, compared with almost two-thirds of program directors (P=.003). The distribution of years’ experience as a director was similar between the two groups.

table1
table2

 

Table 2 compares the survey responses of clerkship directors and program directors regarding their rating of how important it is for medical school graduates to demonstrate each skill or characteristic. Compared to program directors, clerkship directors rated critical reasoning (4.68 versus 4.49), documentation skills (4.35 versus 4.08), hands-on experience (4.56 versus 4.25), medical knowledge (4.42 versus 4.26), and patient-centeredness (4.20 versus 4.01) higher compared to the program directors in the area of whether these skills were required of entering residents (P<.05 for all comparisons). Program directors rated commitment to specialty and creative thinking higher than clerkship directors (P<.05). Other skills and characteristics exhibited similar response distributions between the two groups. Sub-analysis of program director responses by residency type (university versus community-based) and community size of residency did not reveal any important patterns.

Respondents’ mean ratings of how many medical school graduates consistently demonstrate each skill or characteristic are shown in Table 2. Clerkship directors endorsed higher numbers of medical school graduates consistently demonstrating a skill than program directors for all but four professional characteristics.

When looked at another way, the top skills or characteristics ranked by each group differed as well. Both groups valued the importance of honesty, professionalism, and communication skills among incoming residents. Clerkship directors more often rated critical reasoning and hands-on experience as absolutely required of graduates, while program directors highly rated compassion and tolerance.

Finally, there some gaps between what program directors valued and what they felt was delivered by incoming residents. They rated critical reasoning, communication skills, and professionalism as important skills (4.49, 4.68, and 4.71 on 1 to 5 scale, with 5 representing absolutely required). However, those same areas were rated lower in terms of medical school graduates consistently demonstrating those skills. Critical reasoning was rated 3.69, communication skills 3.95, and professionalism 3.97 on a 1 to 5 scale with 1=zero or almost zero demonstrate and 5= all or almost all demonstrate.

 

Discussion

 
 

Clerkship directors rank several areas higher in importance than program directors. Critical reasoning, documentation skills, hands-on experience, medical knowledge, and patient-centeredness were all ranked higher as skills entering residents should have by clerkship directors when compared to program directors. Perhaps clerkship directors are focusing on teaching skills that program directors do not find as important. Program directors ranked commitment to specialty and creative thinking higher than clerkship directors as two characteristics that were more important for entering residents to possess. Because clerkship directors are responsible for teaching all students, not only students entering family medicine residencies, they don’t likely emphasize commitment to specialty as highly. Clerkship directors may focus on teaching skill sets and perhaps not identify characteristics in students that might fit better with family medicine residency or career choice.

In terms of frequency of students who demonstrate skills, all skill areas were ranked higher by clerkship directors as compared to program directors. Clerkship directors may be over-estimating the skill set of graduating medical students. Or perhaps clerkship directors are more removed from fourth-year students and not able to accurately rank some of their skills.

Also, program directors seemed to value certain characteristics more than clerkship directors: commitment to service, commitment to specialty, compassion, creative thinking, empathy, optimism, tolerance. Medical student educators may value skills more than characteristics. On the other hand, personal characteristics rather than skills seem more important to program directors than clerkship directors. Perhaps program directors think they can teach the basic skills that entering residents lack but that certain key characteristics in residents are non-negotiable.

A weakness of the study is that program directors may be recalling all incoming residents’ strengths, not just US medical graduates, while clerkship directors were sampled from US and Canadian medical schools specifically. The overlap of students as judged by the two groups of respondents is unknown. The students rated by clerkship directors are not exactly the same cohort of students rated by the program directors. Another weakness is that this is the perception of the clerkship directors and program directors regarding the frequency with which students demonstrate these skills and characteristics, not a formal test of the presence or absence of these characteristics in the incoming residents.

A future area of focus may include a more formal program for feedback from program directors to clerkship directors. For example, if residents are placed on probation during their residency, should that information get sent back to the medical school from which they graduated? Implications for the future may include a more rigorous focus from family medicine clerkships on professionalism curriculum and teaching communication skills, as those were areas valued by program directors but not rated as commonly seen among incoming residents.

Acknowledgments: This paper was presented at the 2016 Society of Teachers of Family Medicine Conference on Medical Student Education, Phoenix, AZ.

Corresponding Author: Address correspondence to Dr Swofford, University of Missouri-Columbia, Department of Family and Community Medicine, One Hospital Drive, Columbia, MO 65212. 573-882-9097. Fax: 573-882-6228. swoffords@health.missouri.edu.

 

References

 
  1. STFM National Clerkship Curriculum. http://www.stfm.org/Resources/STFMNationalClerkshipCurriculum. Accessed March 26, 2015.
  2. Epstein RM. Assessment in medical education. N Engl J Med 2007;356(4):387-96.
  3. Langdale LA, Schaad D, Wipf J, Marshall S, Vontver L, Scott CS. Preparing graduates for the first year of residency: are medical schools meeting the need? Acad Med 2003;78(1):39-44.
  4. Promes SB, Chudgar SM, Grochowski CO, et al. Gaps in procedural experience and competency in medical school graduates. Acad Emerg Med 2009;16(Suppl 2):S58-S62.
  5. Paolo AM, Bonaminio GA. Measuring outcomes of undergraduate medical education: residency directors’ ratings of first-year residents. Acad Med 2003;78(1):90-5.
  6. Murdoch W, Porcerelli J, Markova T, Saghir H, Bridge P. Incoming resident experience and comfort with procedures designated as “basic.” Fam Med 2012;44(1):47-50.
  7. Dickson GM, Chesser AK, Woods NK, Krug NR, Kellerman RD. Family medicine residency program director expectations of procedural skills of medical school graduates. Fam Med 2013;45(6):392-9.
  8. Kedian T, Gussak L, Savageau JA, et al. An ounce of prevention: how are we managing the early assessment of residents’ clinical skills? A CERA study. Fam Med 2012;44(10):723-6.
  9. Swofford S, Allmon AL, Garrett EA, Hoffman K. Expectations of the competencies of entering family medicine residents: do members of the family agree? Paper presented at the 2011 Society of Teachers of Family Medicine Conference on Medical Student Education, Houston, TX.
  10. Mainous AG 3rd, Seehusen D, Shokar N. CAFM Educational Research Alliance (CERA) 2011 Residency Director Survey: background, methods, and respondent characteristics. Fam Med 2012;44(10):691-3.

From the Department of Family and Community Medicine, University of Missouri-Columbia (Drs Swofford, Allmon, Kruse, and Garrett); and SUNY Upstate Medical University (Dr Morley).


Copyright 2017 by Society of Teachers of Family Medicine