Printed from:

RU4PC? Texting to Quantify Feedback About Primary Care and Its Relationship With Student Career Interest

Andrea L. Wendling, MD; Andrea E. Wudyka, MD; Julie P. Phillips, MD, MPH; Diane L. Levine, MD;
Elie Mulhem, MD; Anne Victoria Neale, PhD, MPH; Christopher P. Morley, PhD, MA

Background: Faculty and residents routinely offer feedback about medical students’ specialty career preferences, yet influence of feedback has not been quantitatively evaluated. This study aims to report incidence and balance of primary care comments heard by medical students and determine the effect of comments on career interest.

Methods: This multicenter observational cohort study used text messaging and online surveys to examine positive and negative feedback about primary care, as reported by medical students in real time between September 2012 and April 2013. Participants from three universities sent short text messages when primary care comments were heard during two 30-day periods; each period was preceded and followed by surveys assessing career interest.

Results: A total of 120 students (86.3% of recruits) participated in at least one texting period; 87 (62.6%) participated in all aspects of study. Overall, positive comments (851) outnumbered negative (616). Total number of negative comments reported per student was associated with a significantly lower interest in primary care career (β=-.04). There was an association between students’ negative-to-positive comment ratios and lower interest in primary care that approached significance (β=-.145) but only became significant (β=-.191) when variables including institution were added to a linear regression model, supporting the hypothesis that culture toward primary care within an institution can influence graduates’ primary care interest.

Conclusions: Negatively perceived primary care comments were associated with lower interest in primary care careers by medical students. The study provides real-time quantitative data supporting the association between feedback received about primary care and students’ career choices.

(Fam Med 2016;48(1):21-9.)

The United States faces a severe primary care physician shortage.1-4 Despite an increase in the number of medical students over the last decade and a modest increase in those choosing primary care careers,5 the United States is not projected to meet its national primary care needs.6-8 Although many solutions are needed to address this problem, medical schools partially bear responsibility to increase student interest in primary care careers.

During training, medical students are frequently asked about specialty career preferences, often receiving both solicited and unsolicited feedback on this complex decision. Students may lose interest in primary care because of critical comments from mentors, peers, and educators.9-12 The hidden curriculum—a set of influences that function at the level of organizational structure and culture13—is believed to influence students’ ultimate career choices even more than formal policies or curricula.14,15 Student interest in primary care begins to decrease soon after matriculation to medical school,16-18 and the choice of a career in primary care may be particularly affected by “badmouthing” of the professions by attending physicians.11,19,20 Surveys of medical students have shown a high reported incidence of negative comments about primary care.19,21-23 Although several compelling studies have evaluated the content of these comments, all have relied on student recall for data collection and students’ own reports of how these comments influenced their career choices.21,24

This study aimed to quantify the number and positive-to-negative balance of comments about primary care heard by medical students, using a novel methodology consisting of short message service (SMS) text messaging, sent from mobile telephones at point-of-contact. Text messaging for data collection has been used in several research contexts and is associated with higher compliance and satisfaction compared to other methods.25,26 However, text message data collection is a novel approach to measurement of reactions from medical students as they move through the educational process. This approach has advantages: participation is simple, and data can be collected immediately, possibly increasing compliance and minimizing recall errors and recall bias.

We also aimed to explore the relationship between the number of comments heard about primary care and students’ commitment to a primary care career. Together these findings will improve our understanding of how the hidden curriculum influences students’ career choices.




This observational cohort study used SMS text messages sent from medical students as the primary data collection method. Medical students from the College of Medicine at SUNY Upstate Medical University (SUNY-Upstate), Michigan State University College of Human Medicine (MSU-CHM), and Wayne State University School of Medicine (WSU-SOM) participated. These schools were chosen primarily for convenience but represent diverse approaches to undergraduate medical education. MSU-CHM is a public, community-based medical school with seven clinical campuses in rural and urban communities across Michigan. Wayne State University, in Detroit, is an urban public institution and the largest single-campus medical school in the nation. SUNY-Upstate in Central New York is a public medical school offering two options for clinical education: a main campus centered around a university hospital and a distal clinical campus with a longitudinal family medicine clerkship. The percentage of 2014 graduates matching into family medicine from each of these three schools ranged from 4.7%–7.9% (2014 national average 8.5%).27

Study Participants

Medical students from all 4 years of training were recruited from the three schools using e-mail invitations, posters, and announcements. The recruitment letter described the project as a study of feedback that medical students receive about pursuing a career in primary care. All enrolled medical students, regardless of interest in primary care, were invited to participate. Using the random number generator in Microsoft Excel 2007, interested students were randomized into three cohorts, which allowed for data collection over a longer time period. Post-randomization adjustments balanced gender, medical school, and class year distribution.


Each student’s cellular telephone number was stored as a data identifier, linking survey responses and text messaging data. Participants’ names were never linked with mobile numbers. SurveyMonkey.comTM was used to distribute survey instruments to participants via email.28

During each 30-day data collection period, students were instructed to text only a single character, either a “+” or a “-”, to a dedicated study phone whenever they heard a positive (+) or negative (-) comment about primary care from an attending or resident physician. Students were encouraged to consider the tone of the statement, as well as the literal content, to allow for correct classification of “backhanded compliments” and sarcasm and to capture students’ individual interpretations of comments. Investigators specifically did not instruct students in what would constitute positive or negative statements, as the purpose was to capture only the student’s impression of a comment. We felt this would better represent the true effect of any comment, regardless of intent. We deliberately limited data to a single character to allow students a method that was quick and unobtrusive—with the goal of maximizing compliance with the data collection method. At the start of data collection and weekly throughout the data collection period, students received a text asking them to respond with the word “test” if participating in the study. The purpose of these “test texts” was to document continued participation even if students heard no relevant comments.

Students who completed all phases of the study received $50. Participants who completed only one phase received $25. The study protocol was approved by the Institutional Review Boards at each participating university.

Survey Instruments

We surveyed study participants before and after each 30-day texting period. The survey instruments were tested during a pilot study conducted at Wayne State University in 2010 and modified for use at multiple institutions across two time periods. The content of each of the four surveys is detailed in Table 1, and instruments are available in the STFM Resource Library.29


Each pre-texting survey (Surveys 1 and 3) asked about clinical or preclinical assignments in the next 30 days. The initial pre-texting period survey (Survey 1) also collected demographic information (age, gender, race, and ethnicity), rural, suburban, or urban origins, and attitudes toward primary care. Attitudes toward primary care were assessed using a scale developed at MSU-CHM and abbreviated for use in our study. This scale was developed using a comprehensive literature review and has been positively correlated with students’ experience on a family medicine clerkship in a small pilot study.30

Students were re-surveyed after each texting period (Surveys 2 and 4). At each post-texting survey, students were given a list of comments about primary care careers that had been reported previously in the literature21-23 and asked whether they heard similar comments during the preceding texting period. Students could also free-text other recalled comments. In addition, at each survey administration (Surveys 1–4), students were asked, “Will you choose a primary care specialty (such as family medicine, general internal medicine, or general pediatrics) for your career?” This question was rated on a 5-point scale (Yes, Probably, Maybe, Probably Not, No).

To simplify subsequent surveys, correlational analyses were used to compare Likert-scaled questions pertaining to beliefs about primary care with the statement “Will you choose a primary care specialty (such as family medicine, general internal medicine, or general pediatrics) for your career?” We decided a priori that if the majority of these questions were significantly correlated (P<.05), then the matrix of questions could be eliminated from the second-round post-survey. The majority of questions pertaining to beliefs about primary care did have a moderate-to-strong and statistically significant correlation with the career statement. Thus, all primary care attitude questions except the career choice question were eliminated from subsequent surveys.

To evaluate reliability and satisfaction with the data collection process, at the completion of each 30-day texting period we assessed respondents’ self-reported compliance with data collection and satisfaction with texting as a method (Surveys 2 and 4). In order to evaluate the degree of recall bias, at the completion of the study we asked students whether they recalled more positive or negative comments about primary care (Survey 4).


Text data were compiled to create a count of total negative and positive comments reported by all participants and a count of negative and positive comments reported by each participant. An individual negative-to-positive ratio was calculated for each participant by taking the participant’s total count across both periods of all negative and positive comments and dividing total negative count by positive count.

Linear regression was used to predict post-study primary care interest. The raw totals of comments per individual, and the individual negative-positive comment ratios, were modeled as predictors of respondents’ self-reported likelihood of choosing a primary care career from the final survey. The change in student attitude toward primary care that was seen with each variable or group entered in the model was reported as the difference from the baseline mean (the z-normalized value) in order to simplify comparisons between groups. Models were estimated using ordinary least squares (OLS) multiple linear regression techniques with assessment of standard OLS assumptions. We incorporated race, ethnicity, gender, institution, information about family origins, year in medical school, and baseline attitudes toward primary care as covariates, to account for other factors that may impact primary care career interest. Covariates were first analyzed for correlation with one another and with the outcome variable and then entered iteratively in groups, in forward stepwise fashion, with each successive model presented. The coding for the variables in the model is detailed in STFM’s Resource Library.31 Analyses were conducted using SPSS v.20.




A total of 139 students participated in at least one aspect of the study (one survey or text messaging data collection period). Of these, 120 (86.3%) participated in at least one texting phase, 88 (63.3%) participated in both texting phases, and 87 (62.6%) participated in all aspects of study (all surveys and both texting periods). The demographics of fully participating students are described in Table 2. At the start of the study, 37% of responding participants indicated an interest in primary care; 32% responded they probably or definitely would not choose a primary care career, and 31% indicated they were not sure. Breakdown of primary care interest by school is outlined in Table 3.

Table 3

The sum of positive comments across all three institutions (851) was higher than the sum of negative comments (616) for the 120 students who participated in either texting period. As Table 4 indicates, when aggregated, the mean number of positive comments was also greater than or equal to the mean number of negative comments, at all three institutions (ratio negative-to-positive comments, in aggregate=0.72). However, when ratios were reported and analyzed individually by student, the mean ratio per student of negative-to-positive comments for two institutions was above 1.00. We believed this individual student ratio best represented individual student experiences and therefore used the individual student ratio, as well as total number of comments per student, in the final OLS models. Standard deviations for number of negative comments, number of positive comments, and ratio of comments were large.


OLS regression modeling was initially performed for both positive and negative comments and ratios. In simple and limited linear regression models, the raw number of positive comments had no statistically significant association with students’ interest in primary care at the end of the study period. Because of this, we included only the number of negative comments and negative-to-positive ratio of comments, omitting positive comments from the analysis.

In the initial, uncontrolled OLS regression model (Table 5), an increasing number of negative responses was significantly associated with a lower level of interest in primary care at the completion of the study period (β=-.04, P=.014). The association between an increasing ratio of negative-to-positive comments and lower interest in primary care initially approached but did not reach significance (β=-.145, P=.084).


We then incorporated race, ethnicity, rural upbringing, gender, institution, family information, and baseline attitudes toward primary care, iteratively, to account for other factors that may impact primary care career interest. The ratio of negative-to-positive comments became significant (β=-.191, P=.028) with the addition of the variables controlling for individual characteristics (race, gender, rural upbringing) and institution. When the pre-participation interest in primary care was added to the model, all other covariates lost statistical significance, indicating this variable had a stronger influence than all other variables. Year in medical school and ethnicity were not significant in a priori testing and were excluded from the analysis.

To summarize, the total number of negative comments per student had a small but significant correlation with a decrease in likelihood of choosing a primary care career of 0.04 (on a 5-point Likert scale) with each negative comment. The correlation with the ratio of negative-to-positive comments per student was larger (-0.145 on a 5-point Likert scale) but did not reach significance until the effects of other factors, including institution, were included in the analysis. However, during the study period, neither effect was enough to overcome the large influence of a student’s initial impression of primary care.

Although all common primary care comments included in surveys (Table 1) were noted by at least some of our participants, three comments were noted by the majority of participants. Over half of participants reported hearing “Primary care physicians have better relationships with patients” (127/222, 57%), “Primary care physicians are needed in rural areas” (138/222, 62%), and “Primary care income is low” (151/222, 68%). At the conclusion of the study, the majority of students recalled hearing more positive comments about primary care during the texting periods than negative comments (more positive 42/102, 41%; more negative 27/102, 26%; about equal 33/102, 32%).

Of the total valid sample, 74.3% indicated they submitted a text every time or more than half the time they heard a comment, similar to previously published compliance with texting as a data collection method.26 Over 90% of respondents thought text messaging was a good method of data collection.




Attaining and retaining student interest in primary care is essential to meet the health care needs of the nation.32 Although the medical school environment has been identified as a factor influencing specialty choice,15,33 we are not aware of any studies using point-of-contact quantitative data to describe the messages students report hearing about primary care. In addition, this work begins to explore the association of such comments with students’ primary care career interest.

Most published literature describing the hidden curriculum has reported a negative bias toward primary care or generalist careers, with students retrospectively recalling more negative input than positive.19,21,22,24 Interestingly, in our study, students in aggregate reported and recalled more positive comments than negative over the study period. However, when the mean ratio of negative-to-positive comments was analyzed per student, more students reported a negative balance of comments, indicating that the excess of positive comments in the aggregate tally was due to an outlier effect. This individual ratio better represents the impact of comments on each student and supports previous studies that have reported more negative input received by students regarding primary care careers.

Our results suggest that negative comments about primary care are associated with lower student interest in primary care. Notably, the association between an individual student’s ratio of negative-to-positive comments and lower interest in primary care reached statistical significance in this analysis only after institution was included. This important finding supports the hypothesis that the culture toward primary care within an institution can influence graduates’ primary care interest.

All of the frequently identified comments described in previous studies were noted by some of our participants, suggesting that these common beliefs about primary care remain entrenched within the medical education culture. In this study, financial concerns were discussed most often as reasons why students should not choose primary care careers. Despite evidence that primary care careers will be financially sustainable for most medical students,34 negative attitudes about primary care physician income pervade the medical culture.35

This study has several limitations. We collected data over a short time period: each data collection lasted only 1 month, and students participating in both data collection periods were enrolled for 6 months. Although we detected an association between more negative comments and declining interest in primary care, the study duration may not have been long enough to overcome the pre-test inclination toward a primary care career, which was the strongest predictor of posttest interest. In this relatively small sample, it is possible that smaller effects of other predictors were also masked by this large effect. Future studies would be useful to follow students longitudinally throughout medical school, measuring the impact of positive and negative comments at different time points.

The study was also limited by the inclusion of only three US allopathic medical schools, two within a single state. Although the results reported here suggest that institutional environment does have some effect on students’ intentions, including a broader range of medical schools and analyzing the prevalence of positive and negative comments based on institutional characteristics, such as campus structure, mission, tuition cost, and public funding, would be valuable in a future study. A comparison group would also be useful, as the act of monitoring comments itself may have affected primary care interest. Due to these limitations, this should be considered an exploratory pilot study; future researchers might consider a longer time-frame for data collection, a larger sample, accounting for institutional variables, and use of more institutions with greater diversity. Despite these limitations, our results suggest that accumulated negative comments may negatively influence a student’s primary care intent.

Medical student specialty choice is a complex decision affected by many variables, one of which is the input of educators, mentors, and peers. By presenting point-of-contact data and exploring associations with career choice, this study provides further evidence that the hidden curriculum has a measurable impact on the future physician workforce. As medical schools acknowledge their social responsibility to meet the workforce needs of the nation, the importance of understanding the cultural biases of medical professionals, educators, and institutions and the impact of those biases on the primary care workforce will become paramount. To effectively develop the primary care physician workforce for this country, we need to send a clear message from all levels, individual to institutional.

Acknowledgments: Funding/Support: This study was funded by a $10,000 Intramural Mini Grant (grant # 98745) from William Beaumont Hospital Research Institute (Andrea Wudyka, PI; Elie Mulhem, mentor). This project was also partially supported by the Health Resources and Services Administration (HRSA) of the US Department of Health and Human Services (HHS) under grant D54HP23297, “Administrative Academic Units,” (Christopher P. Morley, PI/PD); total award amount for AY 2014 $154,765; approximately 95% of this project was funded by non-governmental sources). This information or content and conclusions are those of the author and should not be construed as the official position or policy of, nor should any endorsements be inferred by HRSA, HHS, or the US Government.

Presentations: This study has been presented in various forms at the following national conferences: 2014 Annual North American Primary Care Research Group (NAPCRG) Meeting, New York City, NY; 2014 Society of Teachers of Family Medicine (STFM) Conference on Medical Student Education, Nashville, TN (conference abstract published in Teaching and Learning in Medicine (Melissa Robinson and Michael D. Mendoza. Abstracts from the proceedings of the 2014 STFM Conference on Medical Student Education. Teaching and Learning in Medicine 2014;27(2): 226-232); 2013 Annual NAPCRG Meeting, Ottawa, Ontario; 2013 STFM Conference on Medical Student Education, San Antonio, TX; 2012 Annual NAPCRG Meeting, New Orleans, LA; 2011 NAPCRG Meeting, Banff, Alberta; and 2011 STFM Annual Spring Conference, New Orleans, LA.

Corresponding Author: Address correspondence to Dr Wendling, Michigan State University, Department of Family Medicine, 11133 Summerhill Way, Charlevoix, MI 49720. 231 675-2245. Fax: 231-582-5338.



  1. AAMC statement on the physician workforce. Association of American Medical Colleges. 2006. https://. Accessed February 15, 2014.
  2. AAMC Physician Workforce Policy Recommendations. 2012., https://. Accessed October 18, 2013.
  3. Bodenheimer T, Pham HH. Primary care: current problems and proposed solutions. Health Aff (Millwood) 2010 May;29(5):799-805.
  4. Schwartz MD. Health care reform and the primary care workforce bottleneck. J Gen Intern Med 2012 Apr;27(4):469-72.
  5. National Resident Matching Program. Results and data 2013 Main Residency Match. Accessed September 3, 2013.
  6. Petterson SM, Liaw WR, Phillips RL Jr, Rabin DL, Meyers DS, Bazemore AW. Projecting US primary care physician workforce needs: 2010-2025. Ann Fam Med 2012 Nov-Dec;10(6):503-9.
  7. Jolly P, Erikson C, Garrison G. US graduate medical education and physician specialty choice. Acad Med 2013 Apr;88(4):468-74.
  8. Chen C, Petterson S, Phillips RL, Mullan F, Bazemore A, O’Donnell SD. Toward graduate medical education (GME) accountability: measuring the outcomes of GME institutions. Acad Med 2013;88(9):1267-80. 1210.1097/ACM.1260b1013e31829a31823ce31829.
  9. Rohan-Minjares F, Alfero C, Kaufman A. How medical schools can encourage students’ interest in family medicine. Acad Med 2015 May;90(5):553-5.
  10. Senf JH, Campos-Outcalt D, Kutob R. Factors related to the choice of family medicine: a reassessment and literature review. J Am Board Fam Pract 2003 Nov-Dec;16(6):502-12.
  11. Erikson CE, Danish S, Jones KC, Sandberg SF, Carle AC. The role of medical school culture in primary care career choice. Acad Med 2013 Dec;88(12):1919-26.
  12. Rodriguez C, Lopez-Roig S, Pawlikowska T, et al. The influence of academic discourses on medical students’ identification with the discipline of family medicine. Acad Med 2015 May;90(5):660-70.
  13. Hafferty FW. Beyond curriculum reform: confronting medicine’s hidden curriculum. Acad Med 1998 Apr;73(4):403-7.
  14. Inui TS, Williams WT Jr, Goode L, et al. Sustaining the development of primary care in academic medicine. Working Group on Sustaining the Development of Academic Primary Care. Association of American Medical Colleges. Acad Med 1998 Mar;73(3):245-57.
  15. Bennett KL, Phillips JP. Finding, recruiting, and sustaining the future primary care physician workforce: a new theoretical model of specialty choice process. Acad Med 2010 Oct;85(10 Suppl):S81-S88.
  16. Bethune C, Hansen PA, Deacon D, Hurley K, Kirby A, Godwin M. Family medicine as a career option: how students’ attitudes changed during medical school. Can Fam Physician 2007 May;53(5):881-5.
  17. Morley CP, Roseamelia C, Smith JA, Villarreal AL. Decline of medical student idealism in the first and second year of medical school: a survey of pre-clinical medical students at one institution. Med Educ Online 2013;18:21194.
  18. Mader EM, Roseamelia C, Morley CP. The temporal decline of idealism in two cohorts of medical students at one institution. BMC Med Educ 2014;14:58.
  19. Hunt DD, Scott C, Zhong S, Goldstein E. Frequency and effect of negative comments (“badmouthing”) on medical students’ career choices. Acad Med 1996 Jun;71(6):665-9.
  20. Hauer KE, Durning SJ, Kernan WN, et al. Factors associated with medical students’ career choices regarding internal medicine. JAMA 2008 Sep 10;300(10):1154-64.
  21. Campos-Outcalt D, Senf J, Kutob R. Comments heard by US medical students about family practice. Fam Med 2003 Sep;35(8):573-8.
  22. Hearst N, Shore WB, Hudes ES, French L. Family practice bashing as perceived by students at a university medical center. Fam Med 1995 Jun;27(6):366-70.
  23. Holmes D, Tumiel-Berhalter LM, Zayas LE, Watkins R. “Bashing” of medical specialties: students’ experiences and recommendations. Fam Med 2008 Jun;40(6):400-6.
  24. Kamien BA, Bassiri M, Kamien M. Doctors badmouthing each other. Does it affect medical students’ career choices? Aust Fam Physician 1999 Jun;28(6):576-9.
  25. Goldstein EB, Savel RH, Chorost MI, Borgen PI, Cunningham J. Use of text messaging to enhance compliance with the Accreditation Council for Graduate Medical Education resident duty hour requirements. J Surg Educ 2009 Nov-Dec;66(6):379-82.
  26. Haller D, Sanci L, Sawyer S, Coffey C, Patton G. R U OK 2 TXT 4 RESEARCH?—feasibility of text message communication in primary care research. Aust Fam Physician 2006 Mar;35(3):175-6.
  27. Kozakowski SM, Crosley PW, Bentley A. Entry of US medical school graduates into family medicine residencies: 2013-2014. Fam Med 2014 Oct;46(9):696-700.
  28. Survey Monkey. https:// Accessed November 24, 2013.
  29. Morley C, Wudyka A, Wendling A, et al. RU4PC survey instruments. STFM Resource Library. 2013.
  30. Phillips JP, Charnley I. Third and fourth year medical students’ changing views of family medicine. Fam Med 2016;48(1):54-60.
  31. Morley CP, Wendling A, Phillips J. RU4PC study coding for variables. STFM Resource Library 2015.
  32. Mullan F, Chen C, Petterson S, Kolsky G, Spagnola M. The social mission of medical education: ranking the schools. Ann Intern Med 2010 Jun 15;152(12):804-11.
  33. Morley CP, Mader EM, Smilnak T, Bazemore A, Petterson S, Rodriguez JE, Campbell KM. The social mission in medical school mission statements: associations with graduate outcomes. Fam Med 2015 Jun;47(6):427-34.
  34. Youngclaus JA, Koehler PA, Kotlikoff LJ, Wiecha JM. Can medical students afford to choose primary care? An economic analysis of physician education debt repayment. Acad Med 2013 Jan;88(1):16-25.
  35. Phillips JP. We must make the cost of medical education reasonable for everyone. Acad Med 2013 Oct;88(10):1404.

From the Department of Family Medicine, Michigan State University College of Human Medicine (Drs Wendling and Phillips); Department of Family Medicine and Community Health, Oakland University William Beauont School of Medicine (Drs Wudyka and Mulhem); Department of Internal Medicine (Dr Levine) and Department of Family Medicine and Public Health Sciences (Dr Neale), Wayne State University; and Departments of Family Medicine, Public Health & Preventive Medicine, and Psychiatry & Behavioral Sciences, Upstate Medical University, State University of New York (Dr Morley).


Copyright 2017 by Society of Teachers of Family Medicine