Printed from: http://www.stfm.org/NewsJournals/FamilyMedicine/PastIssues/ArticleView

Motivational Interviewing Workshop in a Virtual World: Learning as Avatars

Marianna Shershneva, MD, PhD; Ji-Hye Kim, MA; Cynthia Kear, CCMEP; Robin Heyden; Neil Heyden; Jay Lee, MD, MPH; Suzanne Mitchell, MD, MS

Background and Objectives: Limited research has been done to understand outcomes of continuing medical education offered in three-dimensional, immersive virtual worlds. We studied a case of a virtual world workshop on motivational interviewing (MI) applied to smoking cessation counseling and its educational impact.

Methods: To facilitate content development and evaluation, we specified desired MI competencies. The workshop consisted of three sessions, which included lectures, practice with standardized patients, and chat interactions. Data were collected from 13 primary care physicians and residents through workshop observation, and pre- and 3-month post-workshop telephone/Skype interviews and interactions with standardized patients. Interactions with standardized patients were assessed by an expert using a validated MI tool and by standardized patients using a tool developed for this study. For 11 participants who attended two or three sessions, we conducted paired-samples t tests comparing mean differences between the competency scores before and after the workshop.

Results: Expert assessment showed significant improvement on six of seven MI competencies. All participants reported learning new knowledge and skills, and nine described incorporating new learning into their clinical practice. Practicing MI with standardized patients and/or observing others’ practice appeared to be the most helpful workshop component.

Conclusions: The evaluated workshop had positive impact on participants’ competencies and practice as related to MI applied to smoking cessation counseling. Our findings support further exploration of three-dimensional virtual worlds as learning environments for continuing medical education.

(Fam Med 2014;46(4):251-8.)

Three-dimensional (3-D) virtual worlds are increasingly viewed by educators and researchers as learning environments with tremendous potential for the education of health professionals.1-4 A virtual world, also known as a 3-D multiuser virtual environment, is a computer-based, simulated multimedia environment, which offers graphical representation of a physical space where real people use avatars, their digital self-representations, to interact with each other and objects.1,5 An avatar may resemble the person’s appearance or look different. Navigated by the person, it makes gestures, walks, runs, sits, jumps, and flies. When the person types, words appear in a chat box on participants’ computer screens. When the person talks the avatar talks, too, and can be heard by other avatars positioned nearby. A virtual environment, which can be anything from a beach to a library, is usually expressed through color- and detail-rich graphics. Emphasizing its realistic nature, the anthropologist Boellstorff6 called it a place of human culture realized by a computer program through the Internet. More than 250 virtual worlds have emerged in the past decade.7

Sharing characteristics with other forms of distance education, virtual worlds allow learners to study at convenient locations, overcome geographic boundaries, and save on travel expenses. Moreover, immersion and social presence associated with virtual worlds make them particularly attractive to educators and learners. Immersion refers to a sensation of being enveloped by and interacting with the virtual surroundings,8,9 while social presence is a sense of being present with other avatars at the same time and at the same place despite geographic distance.9 Immersion and social presence make virtual worlds conducive to experimentation,10 simulation,11 collaboration,12 community-building,13,14 and constructivist learning.12,15

The well-documented challenges for teaching in a virtual environment are technical problems associated with hardware and software requirements, bandwidth, and firewalls,3,15 a steep learning curve to acquire technical skills,15 and the potential for distraction and exposure to inappropriate content.15 Nevertheless, the majority of virtual world educational activity reports3,16-20 show evidence of learner satisfaction with their in-world experience. Then again, limited research has been done to measure outcomes of virtual world educational programs beyond learner satisfaction.2,20

We conducted a pilot study to advance understanding of virtual world continuing medical education (CME) and its educational impact. The focus of this CME activity was to teach a set of motivational interviewing (MI) skills applied to smoking cessation counseling. MI is a collaborative, patient-centered counseling approach aimed at eliciting and strengthening the patient’s intrinsic motivation to change his or her behavior.21 It is an effective approach to help tobacco users make a successful attempt to quit. A recent systematic review22 showed that MI versus brief advice or usual care resulted in a modest but significant increase in quitting. However, primary care clinicians lack MI skills and underutilize this approach.23

 

Methods

 
 

We designed a pilot, mixed methods research study24 to examine a case of a virtual world CME workshop on MI applied to smoking cessation. Our goal was to explore the potential to teach communication skills to physician learners using a virtual world classroom and assess the educational outcomes, including changes in MI skill proficiency and use of MI in clinical practice. The workshop was offered to primary care clinicians and conducted in Second Life (SL). SL is a 3-D virtual world that was created by Linden Lab in 2003.25 With approximately 27 million registered users,26 it is among the most popular virtual worlds.

Multiple triangulation (ie, data and methodological) was used to enhance the completeness and confirmation of study results.27 It was accomplished by collecting quantitative and qualitative data through objective assessment of learner interactions with standardized patients,28,29 workshop observation, and interviews with learners. This study was approved by the University of Wisconsin Health Sciences Minimal Risk Institutional Review Board and conducted in 2011.

Educational Intervention

The workshop addressed the Five A’s best practices for smoking cessation, Ask, Advise, Assess, Assist, and Arrange for Follow-up,30 and MI techniques focusing on developing empathic partnership and eliciting change talk. As a result of this training, clinicians were expected to enhance their knowledge and skills as related to the Five A’s and develop their MI competencies as follows: (1) collaboratively discuss lifestyle choices with patients, (2) evoke patient thoughts and opinions on the advantages and disadvantages of health behavior, (3) respond with, and demonstrate, complete empathy and avoid correcting patient’s opinions or “preaching,” (4) elicit the patient’s motivation, concerns, and values concerning the lifestyle choice, (5) respond to the patient’s core concerns about lifestyle modification, (6) restate the patient’s reasoning and discrepancies between behaviors and goals or values, and (7) roll with resistance. The desired impact on practice included application of MI competencies and integration of practice changes into routines.

Prior to the workshop, the enrolled participants were offered a 1-hour webinar and technical support to help them set up an avatar and acquire basic SL skills. The workshop consisted of three 2-hour weekly sessions in SL. Each session included a presentation, discussions, and practice exercises. Presentation slides were adapted from those used for real-world, face-to-face events.

Two presenters alternated talking in front of the audience. They used slides, MI role-modeling with two standardized patients, and visual learning special effects, such as cigarette butts raining from the sky to demonstrate the quantity smoked in 1 year by a pack-a-day smoker (Figure 1). Participants were engaged in discussion through the SL chat where they typed their questions, answered the presenter’s questions, and shared comments.

A 30-minute exercise during the first session involved participants writing what to say to a patient and presenters providing verbal feedback. For the 75-minute role-play during the second and third sessions, participants were divided into groups of six to nine and transported to separate rooms (Figure 2). Each participant practiced MI skills by talking with the standardized patient for 10 minutes, receiving feedback from the presenter (verbal and through the chat) and other participants (through the chat), and observing how others interacted with the patient (Figure 3). The participant-patient interactions were video-recorded and posted to a blog site. For homework, participants were asked to review these video-recordings and comment on their own video and those of two other participants.

Two program staff members provided technical support during the sessions. Participants could earn AMA PRA Category 1 CreditTM or American Academy of Family Physicians (AAFP) Prescribed credits. There was no charge for participation.

Figure1
Figure2
Figure3
 

Study Participant Recruitment

It was expected that up to 50 primary care clinicians would be recruited to participate in the workshop through websites of the partner organizations, announcements to health-related groups in SL, flyers at the live educational meetings, and emails to former CME participants. We emailed an invitation to participate in the study to all enrolled workshop participants to recruit 12 individuals who were first to agree.

Data Collection and Analysis

Pre-workshop data collection occurred 1–14 days before the first session via Skype, telephone, or both, with a study participant, a researcher, and a standardized patient. First, the researcher asked the participant to state her or his learning goals, describe practice relative to smoking cessation, and self-assess, using a scale from 1=the lowest to 4=the highest, knowledge of the Five A’s best practices, MI knowledge, and MI skills. The interview lasted approximately 15 minutes. Then, the participant was given a clinical scenario and asked to conduct a 10-minute smoking cessation counseling session with the standardized patient.

Post-workshop interviews took place 3 months after the workshop. Participants were given 10 minutes to interact with the same standardized patient and scenario as during the pre-workshop assessment. Participants then underwent a 20-minute interview to respond to the same self-assessment questions as at the pre-workshop interview, and describe any changes in their knowledge/skills/practice.

All interview/standardized patient sessions were audiorecorded and transcribed. We used a validated tool, Motivational Interviewing Skills for Health Care Encounters (MISHCE),31 to evaluate 15 specific MI behaviors, such as “Elicits/addresses patient’s desired health outcomes/goals” and “Develops discrepancy.” A trained member of the research team scored the audio recordings using the MISHCE tool. Each behavior was rated as 0=Deficient, 1=Developing, or 2=Accomplished. Also, immediately after each session, the standardized patients completed an assessment form consisting of nine statements developed by the project team (eg, “The physician asked me to talk about why I make the choices I make about my health”). Each statement was evaluated as 0=Disagree, 1=Agree Somewhat, or 2=Agree. The research team, with an input from the researcher who validated MISHCE,31 mapped MISHCE behaviors and statements in the patient assessment form to seven MI competencies. Then, MISHCE scores and patient scores were converted to the scores for the MI competencies. Paired-samples t tests comparing mean differences between the competency scores pre-workshop and post-workshop were conducted for both the patient assessment and the expert assessment.

Qualitative analysis involved review of interview transcripts, observation notes, and recorded chat to reveal themes relevant to perceived outcomes, educational components that contributed to these outcomes, and factors influencing practice change. This analysis resulted in a case summary for each study participant, followed by cross-case analysis to identify commonalities and differences among the cases.

 

Results

 
 

Participants

It took 3 months to recruit 22 physicians, residents, and nurse practitioners to participate in the workshop. Thirteen of them agreed to participate in the study, including two family physicians, two general internists, and nine family medicine residents (from eight states). The mean number of years in practice for the four practicing physicians was 19. The majority of residents were finishing the second or third year of their residency. Eight were males; four had previous experience using SL; and six attended the pre-workshop webinar. Nine participated in all sessions, two participated in two sessions, one participated in one session, and one did not participate. Data from 11 participants who attended two or three sessions were included in the analyses.

Satisfaction With the Workshop

Based on interviews, participants were satisfied with the workshop. Four participants reflected on SL in particular. Positive comments described SL as “interesting,” “fun,” “exciting,” and “different.” One participant said: “It is kind of amazing that we had people from all over the country and from many places doing the classes [together].” Critiques of SL included the lack of facial expressions/difficulty to read body language, the technical hurdles, and adjusting to use of an avatar.

Based on observations, technical challenges were minimal. They included difficulty finding the auditorium, lost sound, poor internet connection, lack of avatar navigation skills, and difficulty accessing video-recorded role-plays. The number of technical issues was greatest in the first session and decreased in subsequent sessions.

Self-Assessment of Knowledge and Skills

During interviews, participants’ self-assessments of Five A’s knowledge, MI knowledge, and ability to apply MI to smoking cessation identified perceived improvements (Figure 4). Two participants assessed their MI knowledge lower post-workshop than pre-workshop, because they realized there were MI-related facts and techniques of which they were not previously aware. All participants, including the above two, indicated learning something new, such as “how to be less direct and more engaging in the interview technique,” “not being judgmental,” “amplifying ambivalence,” and “rolling with resistance” (quotations from interviews). Several residents felt that they needed additional MI experience with their patients to further develop their skills.

Figure4

Objectively Measured Changes in Competencies

According to expert assessment of pre/post-workshop standardized patient encounters, nine participants demonstrated three or more improved competencies. Mean post-workshop scores for competencies did not exceed 0.833, on a scale with the minimum value 0 and the maximum value 2.000. However, analysis revealed significant improvement (P<.05) on six of seven competencies. When adjusted for multiple comparisons (P<.007, Bonferroni correction), two competencies remained significantly improved: demonstrate empathy/avoid “preaching,” and respond to the patient’s core concerns about lifestyle modification (Table 1).

Table1

Assessment by standardized patient documented that all but one participant demonstrated two or more improved MI competencies at the post-workshop assessment. On a scale with the minimum value 0 and the maximum value 2.000, mean scores for competencies varied from 0.636 to 1.001 pre-workshop, and from 1.091 to 1.304 post-workshop. Pre/post mean score differences showed improvement on all seven competencies, but none was statistically significant.

Changes in Clinical Behavior

Based on interviews, nine of 11 participants reported applying newly learned MI strategies to help their patients quit smoking and make other desired health-related behavior changes (eg, in respect to weight loss). Physicians noted becoming more efficient in motivating smokers to quit. Residents reported using MI with some of their patients, but they acknowledged the need to gain more experience with MI.

For example, a general internist noted that, before the workshop, he relied on “hammering on the health benefits” when addressing smoking with his patients. After the workshop, he noticed himself “reasoning with the patient as to why they should stop.” Another example was a third-year resident who began to use what she had learned with her patients. Third-year family medicine resident:

You’re kind of thinking like here you are, a standardized patient, you know, that sounds interesting, but does that really, and we all said that to each other, like that doesn’t really work in a real patient. But when I actually had a patient, and I started talking to them about it, and kind of doing the same sort of questioning and, you know, the motivational stuff, it was like, oh, wow, this actually works on our patients!

Other participants also discussed palpable practice changes. Family physician:

I’m much more open in interviewing people now. I know exactly what to ask and when, you know, and I think I’m much more confident in that aspect.

First-year family medicine resident:

[MI] has definitely helped me kind of look in the mirror at how I’m kind of encouraging people, and I think it’s changed the way that I’ve done things, not at every visit, but it’s definitely probably once a day changed the way I kind of go about talking to people.

What Contributed to the Achieved Outcomes?

All but one participant attributed their changes in knowledge, skills, and practice to their participation in the workshop. This one participant could not separate the educational impact of the MI workshop from the impact of a previously attended training.

The following educational components appeared to have the greatest influence on the outcomes: practicing MI with the standardized patients, receiving feedback from the instructor and peers, and observing how others interacted with the standardized patients. One participant highlighted the importance of interaction with instructors and peers throughout the sessions, including chat. A few participants reflected on the effective structure of the workshop.

Perceived facilitators of practice changes were seeing patients quit smoking, supportive colleagues and professional community, and availability of resources in the participant’s hospital or clinic. Time constraints were reported as the major barrier for incorporating MI into clinical practice.

 

Discussion

 
 

This study examined a case of virtual world CME on MI for primary care clinicians and physicians in training. Although objective pre- and 3-month post-assessment indicated that participants did not fully develop the MI competencies, it showed a promising progress in the desired direction. Further, participants reported improvements in knowledge, skills, and some aspects of their clinical practice relative to smoking cessation and MI.

These findings were reasonable to expect, given that standardized patient simulations are known to facilitate changes in clinical practice and patient outcomes, in particular, when used to teach communication skills.32 Our findings are consistent with and help to advance prior research. Two virtual world CME studies that used role-play exercises documented positive outcomes, although one study3 did not assess behavior change, and another one20 objectively measured competencies pre-workshop and immediately post-workshop but did not examine the sustainability of changes. Studies of face-to-face MI workshops showed various degrees of success in terms of acquisition and application of skills. However, the researchers33-35 seem to be in agreement in that a workshop alone can help clinicians acquire some MI skills but is unlikely to result in enduring competence, and clinicians need ongoing support over a long period of time, such as feedback, coaching, and follow-up training, to continue skill building, retain competence, and incorporate MI in the routine practice.

Technology challenges and limitations, such as lack of facial expression, were notable but not dominant in our study. We found it interesting that participants seemed to focus on the content and their learning rather than technology when reflecting on their SL experience, which may be a positive sign of acceptance of virtual worlds as a CME environment.

We also argue that 3-D virtual worlds are more than a new technology and concur with researchers36 who encourage reconsideration of what learning means in such spaces. Social presence is realized in a virtual world differently than in the real world. For example, people use their body unconsciously in real life, while virtual world users have to click a mouse or function buttons to create avatar movement, meaning they must remember and order their bodily reactions. How might this influence learning communication skills? It may be both a distraction and an advantage, because virtual world learners become more conscious of non-verbal communication and more intentional in their choices. Another relates to anonymity. When learners do not share their real names, does the anonymity of their avatar create a safer environment for greater risk-taking that can lead to greater learning gains?

As educators work with virtual learners, they may need to adapt their real-world curriculum and instruction to accommodate the limitations and leverage the opportunities of virtual worlds. Converting traditional slides and handouts into activities and demonstrations and using animated visual effects, virtual world video-recording, local chat, and other feasible in-world tools can enhance the instruction but call for innovative thinking and new skills. Rethinking teaching approaches and acquisition of new skills require patience and time, but such investments will allow educators to take full advantage of virtual world affordances, achieve synergy between the environment and content, and enhance teaching-learning transactions.

Study limitations include a small convenience sample, absence of a comparison group, potential bias in expert assessment as the expert knew whether the standardized patient encounter was recorded pre-workshop or post-workshop, use of a non-validated patient assessment form, and shortcomings of self-reports and recall. To mitigate limitations and increase the study trustworthiness, data were collected from multiple sources using multiple methods. Given the limitations, our findings should not be generalized to the broader population of primary care clinicians.

Our study supports further exploration of 3-D virtual worlds as learning environments for continuing education of family physicians and other primary care clinicians. Immersive and bridging space and time, virtual education can help busy clinicians improve their practice. As virtual words evolve and occupy a more central place in education,36,37 it is important for family medicine teachers to engage with in-world learning environments to fully understand their affordances and shape their development to work well for the primary care domain.

Acknowledgments: The authors would like to thank Drs Heidi Moore, Tatjana Petrova, and John Wiecha for their contributions to the study. This study was part of the multi-organizational initiative “Cease Smoking Today” (http://www.ceasesmoking2day.com/). This study was funded by an educational grant from Pfizer and supported by grant 1UL1RR025011 from the Clinical and Translational Science Award (CTSA) program of the National Center for Research Resources, National Institutes of Health.

The content of the manuscript has been presented at the 2011 17th Annual Sloan Consortium (Sloan-C ) International Conference on Online Learning, Orlando, FL; the 2012 37th Annual Conference of the Alliance for Continuing Medical Education, Orlando, FL; and the 2012 CME Congress, Toronto, Ontario.

Conflicts of interest disclosure: Dr Suzanne Mitchell is a non-product speaker for Merck & Company on relationship-centered care.

Corresponding Author: Address correspondence to Dr Shershneva, Office of Continuing Professional Development, School of Medicine and Public Health, University of Wisconsin, 8010 Excelsior Drive, Suite 100, Madison, WI 53717. 608-828-1749. Fax: 608-828-1759. mbshershneva@ocpd.wisc.edu. 

References

  1. Boulos MN, Hetherington L, Wheeler S. Second Life: an overview of the potential of 3-D virtual worlds in medical and health education. Health Info Libr J 2007;24(4):233-45.
  2. Hansen MM, Murray PJ, Erdley WS. The potential of 3-D virtual worlds in professional nursing education. Stud Health Technol Inform 2009;146:582-6.
  3. Wiecha J, Heyden R, Sternthal E, Merialdi M. Learning in a virtual world: experience with using second life for medical education. J Med Internet Res 2010;12(1):e1.
  4. Wood A, McPhee C. Establishing a virtual learning environment: a nursing experience. J Contin Educ Nurs 2011;42(11):510-5. doi: 10.3928/00220124-20110715-01. Epub 2011 Jul 22.
  5. Sivan Y. 3D3C real virtual worlds defined: the immense potential of merging 3D, community, creation, and commerce. J Virtual Worlds Res 2008;1:1-31.
  6. Boellstorff T. Coming of age in second life: an anthropologist explores the virtually human. Princeton: Princeton University Press, 2008.
  7. Association of Virtual Worlds. The blue book: consumer guide to virtual worlds. Available from http://www.calvin.edu/~dsc8/documents/VirtualWorldsBlueBookMay2008.pdf Accessed October 12, 2012.
  8. Nechvatal J. Immersive ideals/critical distances: a study of the affinity between artistic ideologies based in virtual reality and previous immersive idioms. Köln, Germany: LAP Lambert Academic Publishing, 2009.
  9. Dreher C, Reiners T, Dreher N, Dreher H. Virtual worlds as a context suited for information systems education: discussion of pedagogical experience and curriculum design with reference to Second Life. J Inf Syst Educ 2009;20(2):211-24.
  10. Beard L, Wilson K, Morra D, Keelan J. A survey of health-related activities on Second Life. J Med Internet Res 2009;11(2):e17.
  11. Ahern N, Wink DM. Virtual learning environments: Second Life. Nurse Educ 2010; 35(6):225-7.
  12. Mantocani F, Castelnuovo G, Gaggioli A, Riva G. Virtual reality training for health-care professionals. Cyberpsychol Behav 2003;6(4):389-95.
  13. Edirisingha P, Nie M, Pluciennik M, Young R. Socialisation for learning at a distance in a 3-D multi-user virtual environment. Brit J Educ Technol 2009;40(3):458-79.
  14. Wang SK, Hsu HY. Using the ADDIE model to design Second Life activities for online learners. TechTrends 2009;53(6):76-81.
  15. Inman C, Wright VH, Hartman JA. Use of Second Life in K-12 and higher education: a review of research. Journal of Interactive Online Learning 2010;9(1):44-63.
  16. Hew KF, Cheung WS. Use of three-dimensional (3-D) immersive virtual worlds in K-12 and higher education settings: a review of the research. Brit J Educ Technol 2008;41(no. 1):33-55.
  17. Sabus C, Sabata D, Antonacci D. Use of virtual environment to facilitate instruction of an interprofessional home assessment. J Allied Health 2011;40(4):199-205.
  18. Yellowlees P, Cook JN, Marks SL, et al. Can virtual reality be used to conduct mass prophylaxix clinical training? A pilot program. Biosecur Bioterror 2008;6(1):36-44.
  19. Melús-Palazón E, Bartolomé-Moreno C, Palacín-Arbués JC, et al. Experience with using Second Life for medical education in a family and community medicine education unit. BMC Med Educ 2012;12(1):30. (Epub ahead of print)
  20. Mitchell S, Heyden R, Heyden N, et al. A pilot study of motivational interviewing training in a virtual world. J Med Internet Res 2011;13(3):e77.
  21. Miller WR, Rollnick S. Ten things that motivational interviewing is not. Behav Cogn Psychoth 2009;37:129-40.
  22. Lai DT, Cahill K, Qin Y, et al. Motivational interviewing for smoking cessation. Cochrane Database Syst Rev 2010;1. Art. no.: CD006936. DOI: 10.1002/14651858.CD006936.pub2.
  23. Noordman J, van Lee I, Nielen M, Vlek H, van Weijdenc T, van Dulmena S. Do trained practice nurses apply motivational interviewing techniques in primary care consultations? J Clin Med Res 2012;4(6):393-401.
  24. Teddlie C, Tashakkori A. Foundations of mixed methods research: integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks, CA: Sage, 2009.
  25. www.secondlife.com. Accessed November 1, 2012.
  26. Korolov M. Virtual world usage accelerates. 2011. Available from http://www.hypergridbusiness.com/2011/07/virtual-world-usage-accelerates/. Accessed October 12, 2012.
  27. Thurmond VA. The point of triangulation. J Nurs Scholarsh 2001;33(3):253-8.
  28. Koerber A, Crawford J, O’Connell K. The effects of teaching dental students brief motivational interviewing for smoking-cessation counseling: a pilot study. J Dent Educ 2003;67(4):439-47.
  29. Zabar S, Hanley K, Stevens DL, et al. Can interactive skills-based seminars with standardized patients enhance clinicians’ prevention skills? Measuring the impact of a CME program. Patient Educ Couns 2010;80(2):248-52.
  30. Fiore MC, Jaen CR, Baker TB, et al. Treating tobacco use and dependence: 2008 update. Clinical practice guideline. Rockville, MD: US Department of Health and Human Services, Public Health Service, 2008.
  31. Petrova T. Designing an instrument for measuring motivational interviewing skills acquisition in healthcare professional trainees. Auburn University, 2011. Dissertations and theses: full text, ProQuest. Accessed January 19, 2012.
  32. McGaghie WC, Siddall VJ, Mazmanian PE, Myers J. Lessons for continuing medical education from simulation research in undergraduate and graduate medical education: effectiveness of continuing medical education. Chest 2009 Mar;135(3 Suppl):62S-68S.
  33. Miller WR. From the desert: confessions of a recovering trainer/what about decisional balance? Motiv Interviewing 2013;1(2). DOI: 10.5195/mitrip.2013.30.
  34. Söderlund LL, Madson MB, Rubak S, Nilsen P. A systematic review of motivational interviewing training for general health care practitioners. Patient Educ Couns 2011;84(1):16-26.
  35. Forsberg L, Forsberg LG, Lindqvist H, Helgason AR. Clinician acquisition and retention of motivational interviewing skills: a two-and-a-half-year exploratory study. Subst Abuse Treat Prev Policy 2010;5:8. DOI: 10.1186/1747-597X-5-8.
  36. Savin-Baden M. From cognitive capability to social reform? Shifting perceptions of learning in immersive virtual worlds. ALT-J: Research in Learning Technology 2008;16(3):151-61.
  37. Salmon G. The future for (second) life and learning. Brit J Educ Technol 2009;40(3):526-38.

From the University of Wisconsin School of Medicine and Public Health, Madison, WI (Dr Shershneva and Mr Kim); California Academy of Family Physicians, San Francisco, CA (Ms Kear); Heyden Ty LLC, Alameda, CA (Ms Heydan and Mr Heyden); Long Beach Memorial Family Medicine Residency Program and California Academy of Family Physicians, Long Beach, CA (Dr Lee); and Boston University School of Medicine (Dr Mitchell).


Copyright 2017 by Society of Teachers of Family Medicine