AAMC’s Learn Serve Lead Conference: Statistical Significance is Not Everything!

Kristen Jogerst

Surgical Education Research Fellow - Massachusetts General Hospital
PGY-3

The title of this blog post is borrowed from one of the Medical Education Research Certificate (MERC) workshop’s slides. This session led by Dr. Larry Gruppen, PhD reviewed the methodologies behind proper “Hypothesis-Driven Research.” This session also reviewed the importance of asking “feasible, interesting, novel, ethical, and relevant” research questions. The session ended by reviewing the importance of effect size, sample size, and statistical significance within a power calculation. Often in education research, the sample size is small and therefore may not achieve an adequate enough power to detect statistical differences or statistical significance, even though a true educational difference exists. Because of this sample size conundrum, the effect size measurement can show that a difference between two populations of trainees - those who received the educational intervention and those who did not - does exist. The effect size is conveniently independent of sample size and can be very powerful (no pun intended) at detecting meaningful differences in educational interventions.

Conference attendees were greeted with a warm Phoenix
welcome at the annual AAMC conference

During my surgical education research fellowship I am working to obtain my MERC certificate by attending the MERC workshops offered by the Association of American Medical Colleges (AAMC). This month I went to the AAMC’s annual conference to attend two MERC workshops: “Hypothesis-Driven Research” and “Questionnaire Design and Survey Research.” Both sessions were very informative and taught me new ways of thinking about research design and methodology. As I work towards a career in surgical education and education research, the workshops are helpful reviews on proper educational research methodology. They also often serve as dedicated time to work through potential pitfalls in my planned projects with experts in the field. If other medical education community members involved in research are considering attending these MERC workshops, I highly recommend them. If you are like me, you may at one point thought that because you have background training in research methods, you didn’t need further training in education research methods. Having now learned a lot about the different lens through which educational research examines research questions, hypotheses, statistical methods, and analytics, I realize how important this extra training is.

Enjoyed learning a lot about medical education research methods
at the full-day MERC workshop. Also enjoyed reviewing the material at a
local coffee shop during a mid-day break

Beyond the MERC workshops, I am so thankful I attended the AAMC conference this year - it exceeded my expectations. The opening plenary by Bryan Stevenson reinvigorated my passion for medical education and surgical training and was a powerful reminder about the importance of helping those who need it, not just those who can easily access our help. He encouraged all attendees to get proximate to the poor, marginalized, and unequal in order to make our communities healthier. This session, like the MERC workshops, challenged me to think outside of my comfortable framework: both within educational research and medicine.

What are your thoughts? If you had to add a post-it note
to this conference board, what would you write? This conference board
was a helpful reminder to me that health care,
medical education, and education research are all inter-related

He talked about how through hope we can change the status quo, but that being hopeful takes courage and can be uncomfortable. He reminded us that as humans we like comfort, but we have to commit to being uncomfortable. This conference taught me that prior research paradigms might be more comfortable, but they don’t necessarily always fit within the context of education research. Certain career trajectories and day-to-day care for our patients and instruction for our trainees might be more comfortable. If they are, we have to stop and ask ourselves: why are they comfortable? And can we change: our research designs, our teaching methods, and our patient care delivery, to make them more uncomfortable, but also more hopeful for future trainees and for future patients we serve?

I am looking forward to completing the remainder of the MERC workshop sessions to obtain my certificate. But I am also looking forward to attending the AAMC annual conference in the future. I originally attended for the purpose of completing the workshops, but was inspired by the many conference sessions and look forward to going back to learn more next year.

I learned more than medical education research methods during the AAMC conference. I appreciated the reminder that paradigm shifting education, research, and patient care might be uncomfortable, but we have to try. We will not accomplish everything, but we owe it to our patients to be hopeful and to try