Printed from:

Teaching Cross-Cultural Communication Skills Online: A Multi-Method Evaluation

Amy L. Lee, MD; Emily M. Mader, MPH, MPP; Christopher P. Morley, PhD, MA, CAS

Background and Objectives: Cultural competency education is an important and required part of undergraduate medical education. The objective of this study was to evaluate whether an online cross-cultural communication module could increase student use of cross-cultural communication questions that assess the patient’s definition of the problem, the way the problem affects their life, their concerns about the problem, and what the treatment should be (PACT).

Methods: We used multi-method assessment of students assigned to family medicine clerkship blocks that were randomized to receive online cultural competency and PACT training added to their standard curriculum or to a control group receiving the standard curriculum only. Outcomes included comparison, via analysis of variance, of number of PACT questions used during an observed Standardized Patient Exercise, end-of-year OSCE scores, and qualitative analysis of student narratives.

Results: Students (n=119) who participated in the online module (n=60) demonstrated increased use of cross-cultural communication PACT questions compared to the control group (n=59) and generally had positive themes emerge from their reflective writing. The module had the biggest impact on students who later went on to match in high communication specialties.

Conclusions: Online teaching of cross-cultural communication skills can be effective at changing medical student behavior.

(Fam Med 2015;47(4):302-8.)

Cultural competency (CC) is an important curricular component in undergraduate medical education. Previous studies have demonstrated that CC education can improve health professionals’ culturally sensitive knowledge, attitudes, and cross-cultural communication skills1–3 and lead to improved patient outcomes4 and increased patient satisfaction.1 In 2013, the Liaison Committee on Medical Education (LCME) began requiring medical schools to have measurable objectives for developing skills in CC located at identifiable points within their programs.5

In 2009, Tufts University School of Medicine (TUSM) implemented a longitudinal 3-year required cultural competency curriculum. With at least one study demonstrating a positive impact of online CC teaching on student self-assessment,6 we created a 1-hour online teaching module about CC and health disparities in collaboration with Tufts Health Care Institute, targeting clinical students during the family medicine clerkship. This module was combined with an assignment where students practiced using “Problem-Affect-Concern-Treatment” (PACT) questions with 10 of their patients and wrote reflections about their experiences. PACT is a short version of Arthur Kleinman’s explanatory models approach7 to facilitate cross-cultural communication and increase understanding of the patient perspective, which is associated with improved clinical outcomes.8 PACT was adapted by the corresponding author of this paper for use throughout the cultural competency curriculum at TUSM. Using PACT, students ask patients the following:

1. What do you think the Problem is? 2. How is this problem Affecting the rest of your life? 3. What Concerns you the most about this problem? and 4. Do you feel this Treatment plan will help you?

The purpose of this study was to evaluate whether the online module increased spontaneous use of PACT questions by students during a Standardized Patient Exercise. We also examined whether students responded to the module differentially, according to their future specialty choices, and assessed reactions to the PACT questions via qualitative analysis of student narratives.




We designed a multi-method evaluation to assess the impact of the online module and assignment on third-year family medicine clerkship students at TUSM during the 2010–2011 academic year. The family medicine clerkship is taught in 8 blocks at TUSM, and the online module was randomly assigned in four of the blocks during the year. Students received a general description of the study and provided written informed consent. The students were told only that the study involved an evaluation of patient communication, to avoid artificial activation of CC skills. The study was reviewed and approved by the Tufts University Health Sciences Campus Institutional Review Board.

Quantitative Methodology

Faculty blinded to block randomization scored students on the number of PACT questions (PACT score, range: 0–4) asked during a non-graded communication-focused Standardized Patient Exercise (SPE) conducted in the final week of the FM clerkship. The PACT scores, and scores on an end-of-year Objective Structured Clinical Examination (OSCE), of the intervention and control groups were compared using analysis of variance (ANOVA) procedures. Secondary analysis was conducted to investigate the difference in effect of the online module and assignment between students who entered “high communication” (family medicine, other primary care, emergency medicine, neurology, obstetrics and gynecology, physical medicine and rehabilitation, psychiatry, or radiation oncology) versus “low communication” (all other) specialties, in which students from this study later matched. These categories are similar to “patient-oriented” and “technique-oriented” specialty categorizations used in other studies,9,10 adapted to fit the choices made by this cohort of students.

Qualitative Methodology

Students who participated in the online module submitted written reflections after implementing the PACT questions. Students were asked to reflect on their experiences using PACT questions with patients, including an assessment of utility. The student reflections were independently reviewed and coded by two team members through an open-coding immersion-crystallization process.11 Three team members then jointly aligned the code terminology and identified categories of codes within the data. These categories were then grouped into themes of primary importance as identified through consensus by team members (see Figure 1).

Figure 1



The 173 students from the class of 2012 were eligible for the study, with 125 providing written informed consent to participate in data collection. Six students had incomplete data collection, leaving a total of 119 students (60 intervention and 59 control group) contributing data to this study.

Use of PACT Questions

TUSM family medicine clerkship students who received the intervention asked significantly more PACT questions (P=.049) during the communication-focused SPE but had no difference in end-of-year OSCE scores. These results, along with demographic data, are presented in Table 1.

Table 1

Students who received the intervention and who later chose any high communication specialty, family medicine, or any primary care specialty used more PACT questions during the SPE than those who did not choose those specialty categories, and the differences were significant in all but the pooled “Any Primary Care” category. However, there were no significant differences within the control group between students based on later specialty choice. These results are shown in Table 2.

Table 2

Analysis of Student Narratives

The majority of students reflected positively on the PACT questions, and indicated intent to use them in future practice. A small number of students had negative reflections that the question format was either too rigid or not helpful or that the exercise was time consuming. Table 3 displays positive and negative reactions, with exemplar quotes. Beyond these basic student reactions, four salient themes emerged from the written reflections, displayed in Tables 4 and 5.

Table 3
Table 4
Table 5

Theme 1: Communication was the most commonly identified theme among all student reflections. Students felt the PACT model facilitated increased, constructive communication between themselves and their patients.

In Theme 2: Student Development, students indicated that the PACT model guided them toward a more patient-centered focus, including the ability to reflect upon the students’ own biases.

Under Theme 3: Improved Patient Outcomes, students expressed an increased ability to gather important information regarding etiology and treatment barriers after using the PACT model, as well as address patient knowledge gaps and misconceptions.

We found within Theme 4: Improved Patient Experience, that students believed augmented communication helped establish a positive relationship between patient and provider, which left patients feeling “that their needs were listened to and met.”




The online CC module and PACT patient exercise increased the use of PACT questions students asked during the SPE in the family medicine clerkship. There was no impact on end-of-year OSCE scores, although the OSCE was not specifically designed to evaluate cross-
cultural communication skills. A major confounder, and separate finding, is that intervention-group students who later chose what we termed “high-communication” specialties used more PACT questions in the SPE than those who later chose “low communication” specialties. There were no significant differences observed between high-communication and low-communication specialty matchers in the control group. This suggests that students destined for high-communication specialties were more receptive to the online module and exercise. This finding, paired with largely positive reactions in student narratives, indicates that the addition of the online module and PACT exercise was especially beneficial to students who were receptive to improved communication and CC training.

Students more receptive to the use of PACT questions may have displayed greater cultural confidence, as described by Thomas et al.12 Students reflected on issues of personal bias and the need for a lifelong approach to complex cultural issues in their writings. This may indicate that the active use of PACT questions coupled with reflective writing enhanced the building of cultural confidence, as students were confronted with their own ignorance and biases in a concrete and meaningful way. However, this study was not designed to test hypotheses related to this point. The findings above should also be interpreted in light of the fact that this study was conducted at a single institution in a single academic year, which presents a significant limitation.




Online teaching of cross-cultural communication skills can be effective, and further evaluation at other institutions is recommended. In addition, exploration of the receptivity differences between students with different specialty inclinations is warranted.

Acknowledgments: Tufts Health Care Institute provided funding to support the development of the online learning module discussed in this paper. The collaborative evaluation was partially funded through HRSA grant D54HP23297 (Christopher P. Morley, PI/PD).

This paper was presented at the 2014 Society of Teachers of Family Medicine Annual Spring Conference, San Antonio, TX.

The authors thank Evan Glasson, MFA, for assistance with study administration and data collection, and Ms Jessica Mar for assistance with data entry. Thanks to the Tufts University School of Medicine Office of Educational Affairs: Scott Epstein, MD; Maria Blanco, EdD; and Yung-Chi Sung, PhD. Thanks to Tufts Health Care Institute for funding and support to develop the online module Cultural Competence and Disparities in Health Care, especially Ralph Halpern, MSW, and Rosalie Phillips, MPH. The collaborative evaluation was partially funded through HRSA grant D54HP23297 (Christopher P. Morley, PI/PD).

Corresponding Author: Address correspondence to Dr Lee, Tufts University, Department of Family Medicine, 136 Harrison Avenue, Boston, MA 02111. 617-636-2455. Fax: 617-636-3681.



  1. Beach MC, Price EG, Gary TL, et al. Cultural competence: a systematic review of health care provider educational interventions. Med Care [Internet] 2005 Apr [cited 2014 May 28];43(4):356-73. Available from:
  2. Crosson JC, Deng W, Brazeau C, Boyd L, Soto-Greene M. Evaluating the effect of cultural competency training on medical student attitudes. Fam Med [Internet]. 2004 Mar [cited 2014 May 28];36(3):199-203. Available from:
  3. Melamed E, Wyatt LE, Padilla T, Ferry RJ. Patient-based cultural competency curriculum for pre-health professionals. Fam Med [Internet] [cited 2014 May 28];40(10):726-33. Available from:
  4. Lie DA, Lee-Rey E, Gomez A, Bereknyei S, Braddock CH. Does cultural competency training of health professionals improve patient outcomes? A systematic review and proposed algorithm for future research. J Gen Intern Med [Internet] 2011 Mar [cited 2014 May 26];26(3):317-25. Available from:
  5. Liaison Committee on Medical Education. Functions and structure of a medical school: standards for accreditation of medical education programs leading to the MD degree. Cultural Competence Education. [Internet]. 2014. Available from:
  6. Kutob RM, Senf JH, Harris JM. Teaching culturally effective diabetes care: results of a randomized controlled trial. Fam Med [Internet] 2009 Mar [cited 2014 May 28];41(3):167-74. Available from:
  7. Kleinman A, Benson P. Anthropology in the clinic: the problem of cultural competency and how to fix it. PLoS Med [Internet] 2006 Oct [cited 2014 May 28];3(10):e294. Available from:
  8. Kutob RM, Bormanis J, Crago M, Senf J, Gordon P, Shisslak CM. Assessing culturally competent diabetes care with unannounced standardized patients. Fam Med [Internet] 2013 Jun [cited 2014 May 28];45(6):400-8. Available from:
  9. Manuel RS, Borges NJ, Jones BJ. Person-oriented versus technique-oriented specialties: early preferences and eventual choice. Med Educ Online [Internet] 2009 Jan [cited 2014 Aug 4];14:4. Available from:
  10. Borges NJ, Manuel RS, Duffy RD, Fedyna D, Jones BJ. Influences on specialty choice for students entering person-oriented and technique-oriented specialties. Med Teach [Internet] 2009 Dec [cited 2014 Aug 4];31(12):1086-8. Available from:
  11. Borkan J. Immersion-crystallization. In: Crabtree BF, Miller WL, eds. Doing qualitative research (Research Methods for Primary Care) Sage Publications, Inc,1999:179-84.
  12. Thomas SB, Quinn SC, Butler J, Fryer CS, Garza MA. Toward a fourth generation of disparities research to achieve health equity. Annual Reviews 2011 Mar 18 [cited 2014 Aug 4]. Available from:

From the Department of Family Medicine, Tufts University (Dr Lee); Department of Family Medicine (Drs Mader and Morley), Department of Public Health and Preventive Medicine (Dr Morley), and Department of Psychiatry and Behavioral Sciences (Dr Morley), SUNY Upstate Medical University.

Copyright 2017 by Society of Teachers of Family Medicine