Today's interview focuses on how simulation can be used to teach holistic competence. They talked about how in the late 2000s, they recognized that there were several problems with the existing model of teaching students practice skills. Their solution was to simulate real-world problems so that students could practice specific skills, get feedback, and improve their practice. In today's episode, Marion and Toula detail the steps necessary for creating simulations, how they involved field supervisors in the development of client simulations, the iterative nature of improving the simulations over time, and an innovative approach to help students learn knowledge and skills through practice and feedback they call "Practice Friday".
Download MP3 [29:24]
How to use simulation to teach holistic competence
- Simulation can be used in generalist and in specialized practice courses
- Determine if simulation is being used to teach and/or assess
- Consistent with the course learning outcomes for students, develop/identify a few generalist or specialized competencies you would like students to develop and then a) identify what the students would need to know to develop that competence, as well as b) what they will need to demonstrate to show that they are developing that competence
- Map competencies on to a scenario
- Write a vignette (consult with practitioners who work in this area)
- Determine simulation structure and format
- Develop formative and/or summative assessment instruments that may define the competencies in more fine-tuned language
- Involve field instructors
- Train actor(s)
- Implement simulation – variety of ways
- Add a reflection to capture students’ cognitive and affective reactions
- Evaluate simulation
Transcript
[Intro music]Jonathan Singer: Hello and welcome! You’ve found the Social Work Podcast. My name is Jonathan Singer and I’ll be your host as we explore all things social work.
Introduction
Jonathan Singer: Hey there podcast listeners, Jonathan here! Today’s episode of the Social Work Podcast is an interview with Marion Bogo and Toula Kourgiantakis from the University of Toronto Factor-Inwentash Faculty of Social Work. We’re talking about one of the most innovative ways to help social work students learn practice skills. It’s called “simulation” and it’s something that a lot of disciplines have been doing for a long time and that social work is just catching up on.
Today’s interview focuses on how simulation can be used to teach holistic competence. Marion and Toula talk about how in the late 2000s they recognized that there were several problems with the existing models of teaching students practice skills which focus mostly on field supervisors teaching and assessing and providing feedback. And while they’re good intentions, often times it seems like it doesn’t work so well. So what they did—their solution—was to simulate real world problems so that students could practice specific skills, get feedback and improve their practice. Marion and Toula detail the steps necessary for creating simulations, how they involved field supervisors in the development of client simulations and the iterative nature of improving the simulations over time. They also talk about an innovative approach to help students learn knowledge and skills through practice and feedback that they call “Practice Fridays.”
Now, long time listeners of the podcast might remember Marion from episode 94. She and I along with Mary Rawlings spoke about using simulation as part of standardized clinical examinations, which is called OSCE. But today we’re expanding that so that it’s not just about examinations, but about simulation as a teaching tool in and of itself. Marion is—well she’s about as distinguished a faculty member as I could ever hope to have on the podcast. Not only is she the recipient of excellence awards from her university, the social work profession, but Canada. That’s right, the county of Canada recognized her outstanding achievements, dedication to the community and service to the nation by awarding her the Order of Canada in 2014. Now, Toula is also an award-winning educator and scholar in her own right and she serves as the simulation educator at the Factor-Inwentash Faculty of Social Work. And if you want to learn more about Marion or Toula, their publications and amazing accomplishments, please check out their faculty bios. The links are on the Social Work Podcast website.
Now if you’re interested in learning more about simulation and the Objective Structured Clinical Examination that Marion talks about episode 94, you can check out Marion’s co-author text published by the Council on Social Work Education called Using Simulation in Assessment in Teaching: OSCE Adapted for Social Work.
Now, one note about today’s episode. Marion and Toula are speaking directly to social work faculty. Students will appreciate getting an inside peek at one approach to teaching practice skills and measuring competence, but the main audience is faculty. And I’m pointing this out, not so that you turn off the episode now, but because it’s a little different than my typical episodes which focus mostly on students and practitioners. If you’re a student in field, if you’re a field supervisor and definitely if you’re a faculty member, this episode is full of great insight into how to teach practice skills and evaluate competencies.
And now, without further ado, on to episode 120 of the Social Work Podcast, Using Simulation to Teach Holistic Competence: An Interview with Marion Bogo and Toula Kourgiantakis.
Interview
[04:42]
Jonathan Singer: Alright so Marion and Toula, thank you so much for being here on the Social Work Podcast and talking with us about simulation. Why is simulation a good thing for social work educators to do when it comes to teaching competence in practice?
Marion Bogo: Well as you know, social work practice competencies historically have been taught primarily if not exclusively in the field and we know from a lot of research that this presents a lot of challenges because agencies and field instructors are very limited currently in their ability to offer extensive field instruction that involves the actual direct observation of students in their practice performance. Often field education is about conceptualizing client situations and planning interventions and then reflecting on whether or not the interventions were effective. But in the absence of observation we’re really teaching based on students’ recollection and report of what has been transpiring between themselves and the client. With simulation you actually can observe and see the student’s performance.
There was also a very important field summit sponsored by the Council on Social Work Education held in 2014 and one of the strong recommendations that came out of that was that schools of social work need to provide students with baseline, core foundation skills so that when they engage in field practicum learning they’re better able to use those skills and field instructors then don’t have to teach them. I, with my colleague Mary Rawlings at Azusa Pacific University, began to experiment with using simulation to assess competence using the OSCE. We have many materials about that, but then with my colleagues at the University of Toronto we began to become more interested in teaching using simulation. And so we’ve used a similar format to design the simulations that we’re producing that we’re offering for students, a format similar to that used in the OSCE and described in our book on using simulation and assessment.
[07:32]
Jonathan Singer: You know Marion, when you talk about the student reports versus observing their practice behaviors, it reminds me of the process recording assignment which is for the classic field assignment. It’s the classic way the folks in the classroom are able to see what’s going on. But, but that’s not seeing what’s going on. We think about it as seeing what’s going on, but it’s actually just reading the student’s recollection of what happened and that’s very different. For those of us who sort of came up through that model, how is it that you can actually use simulation to teach competence? What are the things that you actually do?
Toula Kourgiantakis: So we have identified with the work that we’ve been doing over the last number of years many steps that instructors, educators can follow to be able to teach holistic competence using simulation-based learning. It is really ideal when you can come up with some consensus across your school of social work in terms of how it’s going to be used and integrated into the curriculum, but if you are starting small and using it in a classroom the first thing to determine is the competencies that you are trying to identify whether those are generalist or specialized competencies and consequently whether you’re using it in a generalist course or in a specialized practice course. It is important to also know whether you’re using simulation to teach the competencies or whether you’re using simulation to teach and assess the competencies. That’s also an important nuance to make because it will require you to go a little bit further if you are assessing in terms of what you’re going to be using.
In terms of the teaching, what’s really important that instructors do is that they are trying to map the competencies on to a scenario. We frequently will consult with the field and with field instructors to see when we are doing the scenarios that we are using scenarios that really do fit with what’s happening in the field. And we have field instructors who will advise us on the vignettes that we’re writing, on the competencies that we are trying to work on and give us their practice wisdom on our competencies and our vignettes that we are working with.
So our next step after we have tried to map the competencies on to a scenario is to really write a detailed vignette that we can work with that really does capture the areas that we would like to have the students work towards. After we’ve developed the vignette we try to think about what the simulation structure and format will be. How many students will be interviewing? How many students will be giving feedback to their peers? When will the faculty member or educator give feedback? Is there any involvement from anyone external, such as field instructors? Will the students be completing any type of reflections? Will the students be completing anything else at any point during the simulation? How long will their simulation take place for, their actual interview? So there are a lot of different things in terms of the actual format and the modality that are really important to think about and to have that well-planned before you actually implement the simulation.
[11:00]
Jonathan Singer: So let me just make sure I’m getting all of this. So the first thing is this competencies, and when you’re talking about the competencies you’re talking about the EPAS competencies, right? The things the Council on Social Work Education says that all accredited schools of social work have to be assessing. And it sounds like this process of having schools agree on practice behaviors and then interviewing the field… it sounds like it would be a great exercise to do anyway. And then you just talked about figuring out who had to be involved in the simulation. Can you talk a little bit more about that, because it sounds like it varies based on the scenario, is that true?
Toula Kourgiantakis: Well if you’re doing it in the classroom certainly it is the faculty member and your students first and foremost. But in the classroom it is also possible to collaborate with field instructors, and it could be collaboration where they are just sharing their expertise with regards to the actual scenario and giving you some information in terms of whether or not those are practice issues that they might see in their setting. But there are also some educational enhancements that we’ve been experimenting with where field instructors are involved more directly. So not only do they share their expertise before the simulation and also debrief with us after the simulation, but they actually attend our simulation activities as well and they share their feedback to students with regards to the practice behaviors that the students are demonstrating. Or not demonstrating.
[12:36]
Jonathan Singer: So you have students going through this scenario, and there’s a simulation based on this scenario. And so how do you know… if they’re getting it? Like, how do you—how are you able to evaluate this?
Toula Kourgiantakis: So that’s a great question, and that’s where it is really important to ensure that you have clearly identified competencies that you have mapped on to a matrix. And that’s how we start out our competencies, is you take the more abstract EPAS competencies and you really break them down into skills, knowledge, practice behaviors, ethics, values, all of the things that we would actually be able to see and that the instructor has provided teaching around and prepared the students for on the day of the actual simulation. So none of this should be completely new for the students, there should be some preparation for this.
And then after you have prepared the matrix—the competency matrix—you then prepare instruments for teaching resources that you can actually use on the day of your simulation and also following your simulation if necessary. Both the instructor can use these instruments and the students or anyone else who is participating in the simulation can also use these instruments. And those can include your vignette—which part of the vignette are you going to give the students? Are you going to have a little summary of your client coming in at intake that you might get if you were at an agency anyway? Are you going to have some sort of feedback form? Feedback form, we call an unstandardized type of questionnaire that you might develop which is really directly linked with your competency matrix that lists some of the competencies that you really want students to demonstrate and that feedback form can be used both by the faculty member, the instructor, as well as peers or students who are evaluating each other during the simulation and this we have found is a really effective way for students who are not directly engaged in the simulation—so those who are observing—to actually become more actively engaged and to also develop learning of the competencies at a different level. So they’re vicariously learning, they’re sitting and watching their peers, and because they do have a form that they need to complete those students actually have to identify the competencies, they need to give you an example of what the competency looked like when their peer used it, or did not use it, they need to identify what some of the challenges were in the interview, how those challenges may have been mitigated, um… those are some of the ways you can actually measure what the students are learning and what they’re not learning.
And something else that we do in addition to the fact that the students are actually writing their feedback, we actually ask them for oral feedback and everyone has to give oral feedback. Not at the same time, but they do have designated moments where they need to give feedback to their peers. And at the very end of the simulation all of the students need to complete some sort of reflection questionnaire. So they do need to reflect on the practice that they were engaged in. And they also need to reflect on the practice that they observed in their peers and they need to talk about that in their reflection questionnaire which again has structured questions that are tied to the competencies and the objectives of the actual simulation itself. The reflection questionnaire also captures some really important areas related to cognitive and affective reactions and that’s part of what we would say is one of the dimensions that permits students to really work on more of their metacompetencies rather than just working on procedural competencies. So procedural competencies being more so the technical skills, what we see, rather than, you know, pay attention also to the areas that we don’t see, so things like self-awareness, their ability to actually manage their own emotions, their own professional judgment in a situation. Those all influence practice behavior and those are things that are much more difficult for an instructor to measure, but having a reflection questionnaire and guidance throughout the entire simulation permits students to really become more aware of those processes and for instructors as well to know what’s going with students when they are engaging in simulated practice.
[17:03]
Jonathan Singer: I really like how you have observers and participants learning at the same time. Um, I think that’s one of the biggest challenges of any training is how do you not make it one-sided, and it sounds like you really thought through that. I can imagine that this is an iterative process, like the first time you use a scenario or you get feedback and then you have to—you’re like, “oh we could have done it this way.” Is that part of the process?
Toula Kourgiantakis: Absolutely. And this work has been at least 10 years in progress with my colleagues at the University of Toronto, but for the last couple of years we’ve also been experimenting with new projects related to simulation and we’ve been tweaking them as we go along. So that is really an important stage in the process that we would highly recommend, is that, number one, schools that feel like they might not be quite ready to go from zero to 10, you don’t have to go from zero to 10. You can really start quite small. And once you do implement one little project you will definitely get a better idea of what works and what doesn’t—and collect data from your students. It doesn’t have to be a research study per se. Anything that you’re collecting that students are giving you feedback on, what they’re learning, how they’re learning, will give you more information about what you can tweak. And you can also try and collect some data from a research assistant that might be helping you, if you have any doctoral students that might be working on the project, if you have any field instructors that are able to look over your materials and give you a sense of what they might do differently. Or what—what… how it resembles or not, what’s happening in the field. That can also be really valuable information to influence the next phase of the project.
[18:58]
Jonathan Singer: So this 10 year long process, it sounds like you’ve been evaluating it, you’ve been doing some research on it. What sort of things have you done and what have you found?
Marion Bogo: About 10 years ago at the University of Toronto with my colleague Cheryl Regehr we began to try to unpack the notion of competence. And this predated EPAS. We interviewed field instructors in great depth to try to understand what were the internal constructs that they were using to assess students along a continuum from excellent students to poor students. And what emerged from these long, intensive interviews was a view of competence that included higher order abilities of a conceptual and affective nature that we called “metacompetencies” that intricately affected the performance of what we have called “procedural competencies.” We then took that conceptualization and started to test it through our OSCE research program and what emerged was a view of holistic competence that involved knowledge, values, skills and the cognitive and affective processes that students engage in to use, or not use, knowledge, values and skills in their complex practice behaviors within client situations. So this view of holistic competence to a large extent was adopted in EPAS 2015 and we then began to conduct research to test out whether that model was robust enough to be used across generalist and specialist social work practice areas. One of the key studies we did was a study on what we have called Practice Fridays.
Toula Kourgiantakis: So I will tell you a little bit about what Practice Fridays is. Practice Fridays is an educational enhancement that emerged about two years ago at the University of Toronto, and what we were hoping to try out is really just providing an extra opportunity for our students to practice outside of the classroom. And what we found is that the more that we were experimenting with Practice Fridays the more that we got more ideas on how we could modify Practice Fridays, how we could add additional Practice Fridays to continue supplementing student learning.
So, Practice Fridays is a voluntary educational enhancement. Students can come. What we find is that most students do come even if they don’t have to come. We do target—we started it with just Year 1 students. Foundational competencies were being addressed, largely. And now we do offer it to Year 1 and Year 2 students. For Year 1 students, again, just generalist competencies that are being worked on, and for Year 2 students it’s more specialized. So for example we have a Practice Friday that is really addressing doing mental health assessment including a suicide risk assessment.
So what we did is because Practice Fridays was new, we really wanted to get a better sense of what students are learning at Practice Fridays and in simulation in general because our Practice Friday structure resembles very much what we do in a lot of our simulations in the classroom. And so we did ask our students who participated in the Practice Fridays in 2015, what are they learning? And then we also asked them how are they learning at Practice Fridays. And what we learned was really interesting. So students told us that what they were learning was… kind of fell into a few different dimensions. Students described that they were learning—that they were acquiring more knowledge about social work practice and they were also acquiring knowledge about different social work concepts, different social work concerns that might be happening, different things about clients and client issues that they might be dealing with. Students described that they were learning about social work practice skills, what they look like, what they feel like when they’re trying them out. Students also talked about learning about themselves. They talked about an increase in self-awareness after doing Practice Fridays. They talked about being better able to manage their own emotions. They talked about having increased professional judgment and also being able to make decisions in a way that felt… that felt that it fit with whatever it was that we were teaching them. What we saw is that what they were describing in fact fit very much with the Holistic Competence Model that my colleague just described, which talked about—which covers procedural and metacompetencies. So students really described that they were learning about all four dimensions about the Holistic Competence Model.
And what we found really interesting was that when the students talked to us about what they were learning, they were also talking about how they were learning. And that’s what we also found was really relevant because not only did it highlight the important things that we need to keep in mind in terms of what students learn but also how we can teach them. So students did tell us that they were learning through three important processes. First, they were learning by the fact that they were actually engaged in practice, but it wasn’t just the practice itself that was improving their learning, it was the fact that their practice was being observed. And it was not just the fact that their practice was being observed, but it was the fact that their practice was being observed and they were receiving focused feedback on the practice that was being observed. They were also observing their peers practice and they were also hearing focused feedback on their peers practice that was happening during the simulation. This was really relevant because we saw that the two were quite connected, and we also saw that the third process was also really connected to this which was students described having… that they also learned through having a lot of reflection. And not only was there reflection, but it was guided reflection. So getting focused feedback permitted them to better understand, to reflect more on some of the issues that they were engaging with in their simulation. It permitted them to reflect on their own affective and cognitive processes, their own assumptions, their own biases. It permitted them to reflect on some of the things that they saw when they saw their peers in practice. It also permitted them to really connect theory and practice to get a better sense of how theory looked when you actually see practice demonstrated in front of them.
So these results showed us that students were learning about all different areas of the Holistic Competence Model, but it also showed us that in order to teach holistic competence we really need to use holistic competence teaching methods. And observed practice, focused feedback, guided reflection were three important processes that are teaching methods that we could be—that we need to use, but also need to keep in mind that these three processes are really interactive and interrelated, and so they really do inform one another.
[26:46]
Jonathan Singer: Well, so Marion and Toula, this has been just a fabulous summary, discussion of simulation. I love how you had the—here’s the issue, and here’s how we address it, here are the steps for creating a simulation, here’s some research that we’ve done. And there’s this Practice Friday which is—sounds like a very cool way of students to come in and enhance their skills, but also is a way for you to get feedback on what was working in terms of teaching and that’s all great.
Now, it could be that somebody’s listening to this and I want more information. Like, this was a nice taste, but how do I get more information, so what are some references or resources for folks?
Marion Bogo: The major reference is the book published by CSWE: Using Simulation in Teaching and Assessing Social Work Competence. In the book we lay out all the steps to developing scenarios, developing instruments, and we also have a number of well-developed case scenarios or vignettes that can be used in a simulation. We also have conducted a lot of research on this and the papers are all listed on our website and will be listed as part of the podcast series.
It’s been a very exciting adventure for us of teaching in this way and we would urge colleagues to try it out. As Toula said, you can start small. And think seriously about developing an education research component if you’re going to continue to use simulation in teaching.
[28:37]
Jonathan Singer: Well that’s awesome. Ok so, Marion, thank you so much for being here again on the podcast. I really appreciate it.
Marion Bogo: Thank you Jonathan! My pleasure.
Jonathan Singer: And Toula it was nice to hear from you, meet you and have you on the podcast as well.
Toula Kourgiantakis: Thank you very much.
[Outro music]
Jonathan Singer: I’m Jonathan Singer and thanks for being with me today for another episode of the Social Work Podcast. If you missed an episode, or have suggestions for future episodes, please visit SocialWorkPodcast.com. If you’d like to support the podcast, please visit our online store at cafepress.com/swpodcast. To all the social workers out there, keep up the good work! We’ll see you next time at the Social Work Podcast.
END
Transcription generously donated by David Viitala, 2018 MSW graduate of the University of Toronto.
References
Assessing Competence Using OSCE Adapted for Social Work
- Bogo, M., Rawlings, M., Katz, E., & Logie, C. (2014). Using Simulation in Assessment and Teaching: OSCE Adapted for Social Work (Objective Structured Clinical Examination). CSWE: Alexandria, VA.
- Logie, C., Bogo, M., & Katz, E. (in press). “I didn’t feel equipped: Social work students’ reflections on a simulated client ‘coming out’. Journal of Social Work Education.
- Katz, E., Tufford, L., Bogo, M., & Regehr, C. (2014online). Illuminating students’ pre-practicum conceptual and emotional states: Implications for field education. Journal of Teaching in Social Work 34, 96-108.
- Logie, C., Bogo, M., Regehr, C., & Regehr, G. (2013). A critical appraisal of the use of standardized client simulations in social work education. Journal of Social Work Education, 49(1): 66-80. DOI: 10.1080/10437797.2013.755377
- Bogo, M., Katz, E., Regehr, C., Logie, C., Mylopoulos, M., & Tufford, L. (2013). Toward understanding meta-competence: An analysis of students’ reflection on their simulated interviews. Social Work Education 32(2): 259-273. DOI: 10.1080/02615479.2012.738662
- Bogo, M., Regehr, C., Katz, E., Logie, C., Tufford, L., & Litvack, A. (2012). Evaluating the use of an objective structured clinical examination (OSCE) adapted for social work. Research on Social Work Practice. 22(4), 428 - 436. DOI: 10.1177/1049731512437557
- Bogo, M., Regehr, C., Katz, E., Logie, C., Mylopoulos, M. & Regehr, G. (2011). Developing a tool to assess student reflections. Social Work Education 30(2), 186-195.
- Bogo, M., Regehr, C., Logie, C., Katz, E., Mylopoulos, M., & Regehr, G. (2011). Adapting objective structured clinical examinations to assess social work students’ performance and reflections. Journal of Social Work Education, 47, 5-18.
Simulation in Teaching
- Bogo, M., Shlonsky, A., Lee. B., & Serbinski, S. (2014). Acting like it matters: A scoping review of simulation in child welfare training. Journal of Public Child Welfare 8(1)70-93. DOI: 10.1080/15548732.2013.818610
Conceptualizing and Assessing Competence
- Kourgiantakis, T. & Bogo, M. (2017). Developing cultural awareness and sensitivity through simulation. In R. Allan & S. Poulsen (Eds.) Cultural Safety in Supervision and Training. New York, NY: Springer International Publishing. doi: 10.1007/978-3-319-64617-6_8
- Kourgiantakis, T., Bogo, M. & Sewell, K. (in press). Practice Fridays: Using simulation to develop holistic competence. Journal of Social Work Education.
- Bogo, M., Rawlings, M., Katz, E., & Logie, C. (2014). Using Simulation in Assessment and Teaching: OSCE Adapted for Social Work (Objective Structured Clinical Examination). CSWE: Alexandria, VA.
- Bogo, M. (2010). Achieving Competence in Social Work through Field Education. Toronto, ON. University of Toronto Press.
- Bogo, M., Mishna, F., & Regehr, C. (2011). Competency frameworks: Bridging education and practice. Canadian Social Work Review 28(2): 275-279.
- Bogo, M., Regehr, C., Power, R., & Regehr, G. (2007). When values collide: Providing feedback and evaluating competence in social work. The Clinical Supervisor 26(1/2), 99-117.
- Bogo, M., Regehr, C., Woodford, M., Hughes, J., Power, R., & Regehr, G. (2006). Beyond competencies: Field instructors' descriptions of student performance. Journal of Social Work Education, 42(3), 191-205.
- Bogo, M., Regehr, C., Power, R., Hughes, J., Woodford, M., & Regehr, G. (2004). Toward new approaches for evaluating student field performance: Tapping the implicit criteria used by experienced field instructors. Journal of Social Work Education, 40(3), 417-426.
- Bogo, M., Regehr, C., Hughes, J., Power, R., & Globerman, J. (2002). Evaluating a measure of student field performance in direct service: Testing reliability and validity of explicit criteria. Journal of Social Work Education, 38(3), 385-401.
- Regehr, G., Bogo, M., Regehr, C., & Power, R. (2007). Can we build a better mousetrap? Improving measures of social work practice performance in the field. Journal of Social Work Education, 43(2), 327-343.
- Regehr, C., Bogo, M., Donovan, K., Anstice, S. & Kim, A. (2012). Identifying student competencies in macro practice: Articulating the practice wisdom of field instructors. Journal of Social Work Education, 48, 307-319.
- Regehr, C., Bogo, M., Donovan, K., Lim, A. & Regehr, G. (2011). Evaluating a Scale to Measure Student Competencies in Macro Social Work Practice. Journal of Social Service Research. 38(1) 100-109. DOI 10.1080/01488376.2011.616756
- Taylor, I. & Bogo , M. (2013iFirst).Perfect opportunity~perfect storm? Raising the standards of social work education in England. British Journal of Social Work. doi: 10.1093/bjsw/bct077
Resources
- The International Meeting for Simulation in Healthcare: http://www.ssih.org/Events/IMSH-2015
- Building Professional Competence: http://research.socialwork.utoronto.ca/hubhomepage?hub=building_prof_competence
- Svenson, O. (1981). Are we all less risky and more skillful than our fellow drivers? Acta Psychologica, 47(2), 143–148. doi:10.1016/0001-6918(81)90005-6
APA (6th ed) citation for this podcast:
Singer, J. B. (Producer). (2018, May 9). #120 - Using simulation to teach holistic competence: Interview with Marion Bogo and Toula Kourgiantakis [Audio Podcast]. Social Work Podcast. Retrieved from http://www.socialworkpodcast.com/2018/05/simulation.html
No comments:
Post a Comment