4. Pass/Fail
Episode Synopsis
On the path to medicine, we’re constantly taking tests: MCAT, shelf exams, step exams, boards, and more. What are these numbers good for? What are they not good for? What is the emotional impact of these tests, and their scores, on medical learners?
Behind The Scenes
We didn’t expect to create an entire episode about shame and testing, but so many of the stories we received were about this topic that it called out for its own spotlight. Here, we hoped to explore not just the pros/cons of using test scores to predict clinician performance, but also the outsized emotional experiences that test scores incite in medical learners, due to the way those scores are used to assign worth and open/close doors to different medical specialties. This episode felt especially relevant to our team, as one of our medical student producers was in the process of studying for Step 2 as we created this episode, and could speak directly to the extreme emotional toll it was exacting on them and so many of their peers.
CONTRIBUTORS
Will Bynum, MD
Corinne April Iolanda Conn, MPH
Uchenna Ewulonu, MD
Alessandro Racioppi
David Savage, MD, PhD
And other healthcare professionals who wish to remain anonymous
PRODUCTION TEAM
Host & Co-creator: Emily Silverman, MD
Co-creators & Research Leads: Luna Dolezal, PhD & Will Bynum, MD
Producer, Editor & Audio Engineer: Sam Osborn
Producer & Assistant Editor: Molly Rose-Williams
Additional Producing: Adelaide Papazoglou
Production Coordinator: Penelope Lusk
Medical Student Producers: Corinne April Iolanda Conn, MPH & Nikhil Rajapuram, MD
Impact Campaign Manager & Assistant Producer: Carly Besser
Executive Producer: Alison Block, MD
Chief Operating Officer: Rebecca Groves
Exeter Team Coordinator: Alice Waterson
Series Illustrator: Beppe Conti
Music: Blue Dot Sessions
SPONSORS
Shame in Medicine: The Lost Forest is made possible in part by the Shame and Medicine project at the University of Exeter, funded by the Wellcome Trust.
The Nocturnists is supported by the California Medical Association, and people like you who have donated through our website and Patreon page.
-
Download the discussion guide for Episode 4, “Pass/Fail" here.
-
The below resources are specific to episode 4, “Pass/Fail.” To learn more about shame in medical culture, you can find recommended resources here.
LISTEN:
Academic Medicine Podcast, Shame Experiences in Premedical and Medical Students.
READ:
Bynum WE IV, Teunissen PW, Varpio L. In the “shadow of shame”: A phenomenological exploration of the nature of shame experiences in medical students. Academic Medicine. 2021.
Bynum WE 4th, Uijtdehaage S, Artino AR Jr, Fox JW. The Psychology of Shame: A Resilience Seminar for Medical Students. MedEdPORTAL. 2020.
Cuneo C. N. Make the Step 1 test for medical students pass/fail. STAT. 2019.
-
The Nocturnists – Shame in Medicine: The Lost Forest
Ep. 4 Pass/Fail
Episode Transcript
Note: The Nocturnists is created primarily as a listening experience. The audio contains emotion, emphasis, and soundscapes that are not easily transcribed. We encourage you to listen to the episode if at all possible. Our transcripts are produced using both speech recognition software and human copy editors, and may not be 100% accurate. Thank you for consulting the audio before quoting in print.
Emily Silverman
You have this dream, kind of a common dream, where you're in a nondescript building, nothing on the walls, no windows or clocks to tell you the time of day. You walk through a maze of hallways toward a security guard. He pats you down, removes all your belongings, and leads you to a computer terminal. The walls are gray, your nose burns with the smell of cleaning solution. As you sit down at this computer, you look around and realize there are other people sitting at computers, dozens of people, all hunched in front of these screens. They're all clicking their mice. No one's speaking. There's a set of headphones on the desk. You put them on.
At the top of your screen, you notice a timer, counting down. And then you remember, you’re taking a test. And not just any test–a test that will determine your future–what you can do and who you can be. Have you been studying for this test? Studying enough? Suddenly you feel an urge to escape, but you can't. You're locked in and time is passing. If you don't start now, you'll never catch up. So you do the only thing you can do. You start the test.
Computer voice
Complete.
Emily Silverman
You're listening to Shame in Medicine, The Lost Forest. I'm Emily Silverman. How do your test scores affect your self image–who you know yourself to be and what you're capable of in the future? There's been a lot of discussion recently about the utility of these tests, and the cost of placing such an emphasis on sky-high test scores. But are we talking enough about the emotional impact of these tests? How does the shame associated with testing carry forward into the profession?
David Savage
My name is David. I'm a hem onc fellow in Southern California. During medical school, one of the highest stakes tests that anybody takes is the USMLE Step 1. It's high stakes, because the score you get in some ways determines the kinds of specialties that you can do and sometimes the ones that you won't ever be able to do. Our school, like all the other ones in the country, put a lot of stress on this test. I studied really hard for it, especially the six months before the exam. But I was so stressed out about this test that, not only did I delay the test a couple of times, but I had a really hard time sleeping the night before the exam. So much so that when I woke up the next day, I almost thought that I should cancel my test or delay it further because I was so exhausted. But I had simply run out of time to continue to delay my test.
So, in tears, I drove across town–I was in Houston, Texas at the time. I had worked so hard to study the best I could for this test! I'm not generally a good standardized exam test-taker. And so that made me extra stressed about an exam that for all practical purposes was going to determine my destiny. I called my mom on the way over to the test and she said, “No matter what happens, you're gonna get to be a doctor.” And so I went in and bit the bullet, took my test. On every break, I would go to the bathroom and splash water on my face and try to, you know, do whatever I could to make myself feel more energized and I just made it through the test the best I could. But when I left the test day I was just worn out. And I had these constant feelings of dread and regret, like, I had made a big mistake by taking my test, as tired as I was.
On the day that test results came, my peers opened up their portal to see their score. And you could tell some people had done as well as they had hoped. There was a couple of people that printed up their score reports and called their family and said, “Hey,” you know, “I'm going to get to be a surgeon, or radiologist,” or whatever specialty they wanted to do, because they had gotten a score high enough to keep that door open. For me, I was hesitant all day to look at my score. When I finally did. I was happy I passed and I was happy I did better than expected, all things considered. But it definitely was not as good as I had hoped.
Well, for the year that followed, I just tried to put this at the back of my mind and work through my clerkships. But every time I had a clerkship that I enjoyed, I had to ask myself, you know, ”No matter how much you enjoy this, is this something that you won't be able to do because one particular test score is not high enough?” And so I had a lot of shame because of that. And every time since then that I've underachieved at something, I had to ask myself, “Is that because you're really not cut out for this, as demonstrated by your Step 1 score?”
So I think that, coming full circle, now that I'm board certified and I've shown myself again and again that I can pass these tests, I know that I'm better than one test day or one exam score, and that there's a whole lot more to being a physician than the high-stakes exams that we take for certification. But that being said, I think that those high stakes exams are one of the most stressful parts of what we do. And I just hope, for all the trainees that I mentor in the future, I can give them that perspective. So that no matter what happens on any given exam, they will know that they will always be better than any one test score.
Emily Silverman
Will, do you remember taking your step exams?
Will Bynum
You know, how could I forget? That was a singular event in my medical education and among the singular events in my life.
Emily Silverman
And for those of you who don't remember Will, he is a family medicine physician at Duke who studies shame experiences in medical learners.
Will Bynum
In fact, I was in, I guess, my second year of medical school when I was preparing for Step 1. And I developed a, what I thought was something stuck in the back of my throat, over the week or two or three before the test. I mean, it was really concerning. And I, I went to an ENT doctor. And he essentially laughed me out of the room, after I told him I was preparing for the Step exam, and diagnosed me with something called globus hystericus. And I looked up the definition of this right before we got on. The definition of globus hystericus from medicinenet.com, whatever that is: “Globus hystericus is a symptom of some physical disorders, as well as a classic sign of hysterical neurosis, that suggests a physical disorder but is instead an expression of a psychological conflict or need.”
And thinking back on my globus hystericus preparing for Step 1, it wasn't just stress-mediated, there was a psychological conflict. And that conflict was the tension between recognizing that this test, filled with a bunch of multiple choice questions much of which test obscure knowledge, was going to ultimately shape me as a person. And who I could become as a person, what type of medicine I could practice, the difference I can make, all came down to this one test. Or that's the message that I was given, or the narrative that was communicated to us as medical students.
Emily Silverman
I decided, as a medical student, that I was going to make it my mission to annihilate this test. And I remember studying in North Carolina. My parents have this little condo, and I would wake up every morning and sit at this high-top table in their condo and study for, like, twelve hours. And I developed a muscle cramp in my foot. It was in my right foot. I would just be sitting there studying and then my foot would just curl into a “C”, like, pretty uncomfortable. So I’d have to rub it out. And then it would go away for a few hours, and then it would curl back into a “C”. And then after the test it went away. And for the listeners, I did this for six weeks straight. Will, can you explain, what is the USMLE Step 1, 2, 3 exam?
Will Bynum
So, USMLE stands for United States Medical Licensing Exam, and it's, in its origin, a test that ensures to the public, patients, regulators that we have acquired the knowledge necessary to practice medicine safely. The way it's actually used, though, is much different in medical education, in that, because it's a standardized piece of information–one of very few that matches or allows comparison among all students in the country–it's used for high stakes decisions. And it provides a very convenient marker of that potential success. Problematically, it was never designed to do that. And there's very little, if any, evidence that says it's an effective or valid marker of that future potential.
Emily Silverman
So of all the Step tests, the eight-hundred-pound gorilla is the USMLE Step 1. That's the first test. And some of the questions are good, and they actually test for understanding, for concepts. “Can you identify this acid base disorder?” “Can you look at a set of lab results and interpret them correctly?” But some of the questions are really fringe and they test minutia. It presents a question and as you're reading it, you're like, “Oh, I know the answer. The answer is ‘x’.” But then the thing that they ask isn't for “x”. They'll be like, you know, “What is the enzyme associated with ‘x’?” So they take you like two steps beyond, I think, what maybe you need to know, into this place of trivia.
Will Bynum
The postscript to the test is also one of intense anxiety and self-doubt. I mean, Emily, do you remember walking out of the test, and in the day or two after, you remember these questions that you weren't sure about? And then went and looked it up? And you've, you've got the answer wrong, and you're like, “Shit!” And then you start thinking, “I got this other question wrong.” And then you start talking to your friends, “What, what did you put for this and that?” And the whole time, not only are you panicking, but you're, you're in this place of complete lack of control–over the test, but, way bigger than that, what it's going to do to your future.
Emily Silverman
So, Will, you're a residency program director, and you get hundreds and hundreds of applications every year for a handful of spots in your residency program. Do you find these objective test scores to be useful in whittling down the pool of applicants?
Will Bynum
Here's the thing about objective test scores. They're actually necessary in many ways. And so we can't, we can't sit here and talk about eliminating tests in order to address shame in medical education, because tests are always going to have a role. And they have an important role. As a residency program director, I do go through a lot of applications and we have to have ways to manage those applications. But, more important, we have to have ways to try and predict who's going to be a good fit in our program, and who has the potential to navigate the challenges of that program. A number is one piece of information. It's an important piece of information, because it does tell us some things about the person.
But it's the way in which we use it that's really, can be problematic. So if I use it to make a yes-no decision on a person without considering anything else about that person's application–about their capacity to be a competent physician, about their empathy, et cetera–I am narrowing our understanding of that person, and our representation of that person, down to a single number that's highly flawed in its nature. So the numbers have to stay. What we have to really examine is how we're using them. And way more important than that, how our use of that is affecting people emotionally, particularly in the way that they see themselves and understand themselves, and the way that they believe the system sees them as potentially just a number.
Emily Silverman
So recently, the USMLE Step 1 was moved to a pass/fail grading model. Do you think that signifies a positive step away from numbers and stress and toward a more holistic or at least a more simple review process?
Will Bynum
Step 1 going pass/fail is a great thing. It's not going to solve all of our problems, and it's probably not going to solve many of our problems. The way that we use Step 1, and will continue to use other objective measures, is more of a symptom of an underlying condition. And that condition is our extreme reliance on scores as markers of worthiness, frankly, in our profession. And this goes well beyond the medical education and walls of our hospitals. The real problem in medicine is that eventually the scores go away. The practice of medicine has no scores at all. Emily, when, in your, in your practice, when's the last time you received a clear score that told you how you were doing?
Emily Silverman
Never. And I think a lot of young attendings run into this problem, which is that, as a college student, you want to climb the ladder to medical school, as a medical student, you want to climb the ladder to residency. As a resident, maybe you want to climb the ladder to fellowship or to your dream job. But then, once you're an attending, you look up to grab onto the next rung of that ladder. And there kind of is no rung, and you're left sort of free-floating at the top of this ladder. And it's up to you where to climb next.
Will Bynum
We have to find other ways to know that we are good enough–to know that we do belong. And that's great. That's normal. It wouldn't seem so hard if we weren't so reliant on scores up to that point. We also may need to give ourselves as a community more credit in our ability to create competent, empathic, highly qualified physicians, without being so obsessed with, and reliant on, numbers to prove to ourselves that we can do that.
How many people are we missing because, because we have so few routes to the top? So can we create a new set of ideologies around scores? Can we find ways to responsibly use them, and to recognize and be honest about their limitations and then find additional ladders that allow us to work around those limitations, as opposed to trying to cram everybody up the same one, with, with their self-worth and sense of self hanging in the balance?
Anonymous
When I started medical school, I experienced a whole lot of stress, anxiety, insomnia. I remember those first few months of anatomy, I barely slept, my heart would be racing at two a.m. Just like, “Do I belong? Am I really supposed to be here?” It was awful. So when it came to trying to take my board exams, I really struggled. Took me six months to study for Step 1. I barely, you know, was above passing, but low. So when it came to take Step 2, that same shame of, like, not knowing things, and that I was learning things for the first time, because I just was in a fog. I panicked. I had a panic attack. Never had panic attacks before medical school. Worst timing is, of course, when you're trying to take Step. And I had one. I got my exam results two, three weeks before we were supposed to submit our residency apps, and I found out I failed by two points. It was the most heartbreaking. It was just like failure, on top of failure, on top of failure that just–I made the decision to push out residency and, you know, take some time off to study for Step, and did well, and it was good. But, you know, there was just so much shame that I was carrying around, of, like, being a failure and, you know, just not catching up quickly enough.
And there's actually a couple of advisors who really turned it around for me. They shared their own experiences of failing different exams and they were there. They were advising me, they were doing incredible things, gave me hope that, you know, I can, I can do it, too. I learned to accept my failure and mistakes, just as a part of learning. And I'm going to carry it forward, something that can help me relate with other learners who are struggling. And maybe it'll help, I know, hopefully help prevent someone from feeling so horrible that they feel like they can't exist anymore. I'm gearing up for residency interviews right now, and I can't share this raw story with them. Gotta sugarcoat it, gotta focus on my strengths and my resilience. When I walk across that stage, I think what I'll remember the most about this journey is just the struggle to overcome my inner critic, my inner shamer. It's been hard.
Alessandro Racioppi
It was a warm spring day in Chapel Hill, North Carolina. And I can still recall the cherry trees blooming all around me during my drive to meet with my new pre-medical advisor for the post baccalaureate program that I had just joined. I recall as I entered the office building, the white tile ceilings, the bright lights and the gray desk, which surrounded me, I was excited to discuss how I should structure my coursework in preparation for medical school application, and really eager to absorb any advice that this new mentor may have for me.
So, Dr. Jones began our meeting by telling me that she had already looked over my transcripts, and that this would be a quick and straightforward conversation. “This is not a career for you,” she said. The words were as unexpected as they were hurtful. I’d always seen myself as a caregiver, someone that was there to help others throughout the most difficult times of their lives. Someone that would work to solve complex cases and help patients in need. Someone that was eager to learn in a field of lifelong learning. “Your GPA would have you screened out of all U.S. medical schools. At most you could look into international medical programs, but why bother.” And as she kept talking, I was still trying to process her words as everything was moving in slow motion around me.
This professional mentor had just told me that I will never be a doctor. And she went on to quote the current AMC GPA averages for matriculated medical students as of that year. “You're not even close to this range, so you will be screened out at every school.” The more she continued classifying me as a number rather than a person, the more I felt like a barcode, much like the barcodes that you scan at a Walmart to determine the value of the product you're hoping to purchase. More specifically, I was a 3.1. That was my undergraduate GPA. That is who I was.
At the end of that meeting, I left saddened, confused, and, most importantly, lost. You see, a barcode may present you with a standardized value but it tells you nothing about the journey of the individual. Nowhere on that barcode did it say that early in my undergraduate studies, I was diagnosed with cancer, something that affected me, my mental health, my grades significantly at the time. Nowhere on that barcode did it say that I am an immigrant, someone who is able to adjust to a new culture thousands of miles away from home. Nowhere on that barcode was my humanity considered, nor my potential to help others.
And, although there were certainly areas of improvement that I was eager to, and should have, worked on, this advising meeting was never about that. We did not discuss the ways in which I could improve the coursework that had been impacted by my health. But, rather, the entire conversation revolved around trying to have me change my mind about attending medical school at all. I was shamed for being a 3.1 who even dared to consider attending medical school. Although this conversation caused me to associate my self worth with my past grades and performances for quite some time, I'm proud to share that I will finally be starting my first year of medical school in the fall of 2021.
Checkout counter sounds
(Indistinct) I tried the barcode... Can I try? Did you write down the code? Yeah, I did. There you go.
Uchenna Ewulonu
It went like this: “Dear Doctor Uchenna C. Ewulonu, The American Board of Pediatrics regrets to inform you that you were unsuccessful in passing the pediatric board certifying exam.” That was the opening one-liner on my score report, just beneath the word FAIL that was written in aggressive all-caps and was printed in bold, just to make sure that nothing was lost in translation. I remember when I read those dreaded words, this feeling that my stomach was dropping into this endless freefall, because everything I had worked for seemed to be crashing down around me.
And, of course, to make matters worse, I was at work when I received this news. It was about 8 am. It was December 6, 2016, about six months into my new gig as a bright, shiny new attending physician. I remember I was sitting at a computer in the physician work room, when one of my colleagues turned around from her computer across the room to inform me, “Hey, board scores are posted.” I remember she said it so nonchalantly, clearly because she had passed and boards were officially a thing of the past for her. But, unbeknownst to her, this was actually the beginning of an all-encompassing shame spiral that would take me fifteen solid months to crawl out of.
But just a few moments later, my colleague, who had turned to me in the first place, turned back around and said, “So, what are you going to do with your board study material?” She was asking was I going to sell it to one of the interns, or take the more philanthropic route and gift it to one of the trainees, instead. I remember, in that moment, I was racking my brain for what I should tell her, how I should respond. And of course, I ended up just lying. I said I would give it to one of the residents for free.
I just remember feeling like I had to lie, in that moment, because the guilt and shame and this debilitating inadequacy was surrounding me because I failed boards, and was threatening to swallow me whole. I didn't want to imagine what my colleagues or, God forbid, my superiors would think of me if they learned the truth about my short-coming. So I felt obligated to feign success, at a minimum to help me get through rounds and the rest of the work day. But in reality, I knew that I was going to hide the secret for more than just a day. I knew I was going to just bury this secret alive, hoping and praying that, by hiding my failure, it would prevent this deep-rooted impostor syndrome that I have from growing the sturdy, unshakable legs it's always wanted to stand on.
So, in that moment, like any other thoughtful, mature adult, I swallowed my shame. And I allowed it to tear me apart from the inside out. I realize now that internalizing that shame only amplified the deep-seated feeling of not belonging that I had. I had all of these buried thoughts of, “I'm not good enough. I'm not smart enough. I don't actually belong in this space.” And then there's the whole “failing while black” situation. This is exactly what society expected of me. And now I've proven all the naysayers right.
And then, even after I passed boards the subsequent year, I still refused to divulge my dirty little secret if I didn't have to. But the harder I tried to not only hide, but to try and bleach out this dirty little secret that marred my otherwise pristine, crisp white coat, I realized it wasn't going anywhere. It did, of course, fade over time, as most marks will. But it's forever woven into the fabric of who I am as a physician. It's just a part of my story, now, it's a part of my journey. But it's only now that I'm beginning to realize how beautiful that actually is.
Fun fact: I actually have another board exam coming up because pediatric hospital-based medicine is officially a subspecialty of pediatrics. So I get to study all over again and sit for another six-hour exam. But this time, this time if I fail–believe me, I'm praying to every god I've ever heard of to ensure that I don't–but if I do, if I fail again, this time I'm telling everybody. I refuse to succumb to this undue shame that terrorizes my humanity, these shackles of perfection that imprison so many physicians like me. If I fall short again this time around, this time I'm just going to accept reality and let the truth set me free.
Robotic Voice
Process. Processing. Process.
Corinne April Iolanda Conn
I, um, have a motto in life that I kind of developed because of my, um, feet. And the motto is, “Step, by step, by step.” I was born with skeletally interesting feet that I have subsequently had to have bunion surgery for, basically being told by the podiatrist that I got, kind of, born with the feet of a seventy-five-year-old on a twenty-five-year-old body.
So, I always had trouble finding the right shoe that would fit my feet. And I remember the first pair of shoes I ever really invested a lot of money in were these Nike Low Dunks–and I bought them my first year in undergrad–that let me engrave something on the side of them. And they would let me use eight letters, I remember. I thought about putting my name, and then was, like, “I don't really want to be that, that person with my name on my shoes.” So, I came up with this “step by step by, ” so that when I walked, when I walked I was feeling like I was going step, by step, by step, by step. And I had no idea that ten years later, I will be taking this exam, these series of exams, called Step 1 and Step 2. And apparently there is a Step 3.
Step 1 came for me like the boogeyman. I watched it come for all of my friends. It came for us in different ways. It triggered all of our deep insecurities. And this platform we use is called “UWorld.” I hate that name. Because it feels like, “This is your world. This is the only thing in your world that matters, where the score on that performance dictates the rest of your career.” Or at least that was the myth I had internalized. And what that exam required of me was to take everything that I prioritized, and put it on hold because I would work my butt off, ten to thirteen hours a day, to become the computer that was necessary to regurgitate what they want me to know. And my scores on those practice exams would go up maybe ten points if I was lucky. I had started off this path, believing that everything I did could be done, taking things step, by step, by step, and by the time I started studying for Step 1, the mantra that had come into my heart and my mind was, “You're not good enough.”
Oh, I knew I was good enough for my patients. I am good enough for teams. I am good enough for the entire profession of healing. But to become a physician in this country you need a score on Step 1. I felt like medicine had taken my soul and tried to push it through a keyhole. And the shame I have felt around this test being the thing that got to me, this test being the obstacle that made me feel spiritually poor, the shame I feel around being weak enough to be made to feel “less than” by something that was never designed to hold all of me with any amount of compassion. I pray I will have only turned that into resilience.
Emily Silverman
Thanks for listening to Shame in Medicine: The Lost Forest. Next week's episode is called, “Indoctrination,” and looks at how we use shame as a teaching tool in medical education.
To contribute your reactions and reflections to our shame series, and to access additional resources like transcripts, discussion guides, and more, please visit our website at thenocturnists-shame.org.
Shame in Medicine: The Lost Forest was co-created by me, Emily Silverman, as well as Will Bynum and Luna Dolezal. Our producers are Sam Osborn and Molly Rose-Williams, with additional producing by Adelaide Papazoglou. Sam also edited and mixed the series. Thanks to Medical student producers: Corinne April Iolanda Conn and Nikhil Rajapuram. Production Coordinator: Penelope Lusk. Impact campaign manager and Assistant Producer: Carly Besser, and Exeter team coordinator Alice Waterson. The Nocturnists Executive Producer is Ali Block, and our Chief Operating Officer is Rebecca Groves. The series illustrations are by Beppe Conti, and the music comes from Blue Dot sessions.
Shame in Medicine: The Lost Forest was made possible in part by the Wellcome Trust and the Shame in Medicine project at the University of Exeter. The Nocturnists is supported by the California Medical Association, a physician-led organization that works tirelessly to make sure that the doctor-patient relationship remains at the center of medicine. To learn more about the CMA, visit cmadocs.org.
Our show is also made possible with donations from listeners like you. Thank you so much for supporting our work in storytelling. If you enjoy the show, please follow us on your favorite podcast app so you don't miss an episode. While you're in there, you can help others find us, by leaving us a rating and review. To contribute your voice to an upcoming project or to support our work with a donation, visit our website at thenocturnists.com.
I'm your host, Emily Silverman.
See you next week.