The 2022 NBME Invitational Conference for Educators (NICE) gathered more than 250 medical school faculty from 140 allopathic and osteopathic medical schools to gain new skills related to assessment and network with other faculty attendees. Many of the workshops, sessions and presentations at this year’s conference focused on considering diversity, equity, and inclusion (DEI) within assessment practices.

“Assessment practices that promote DEI need to be embedded throughout the development process. NBME continues to investigate and adopt best practices that lead to fair and equitable assessments. As we learn more, we look forward to sharing our findings with others.”

YE TONG, PhD
Senior Vice President, Assessment Operations

Mitigating Bias in Assessment

NICE attendees had the opportunity to learn about mitigating bias in assessment through participating in roundtable discussions led by NBME researchers and joining a panel discussion that featured the Strategic Educators Enhancement Fund (SEEF) Fellows. The SEEF Fellowship supports faculty in the early stages of their career to develop skills in medical education and assessment research. 

During the panel discussion, Amanda Clauser, EdD, NBME Vice President, Psychometrics and Data Analysis shared some of the proactive steps NBME takes to avoid bias and support fairness in testing. Some of these steps include:

• Diversifying test content to represent the broad health care provider and patient populations seen in care settings.
• Working closely with subject matter experts in test design activities to ensure the depth, breadth, and scope of the content limits potential biases.
• Utilizing statistical tools to look for and address potential sources of bias as well as ensure no examinee groups are advantaged/disadvantaged.

The SEEF Fellows shared some of the process behind their research, which centers on mitigating the effects of gender bias in the assessment of professionalism. They also acknowledged some of the challenges with assessing professionalism, including the high-stakes and subjective nature of it. 

Incorporating Patient Characteristics in Test Items

NBME Test Development professionals facilitated a workshop in which attendees learned and practiced how to construct quality test items based on lessons from the NBME Item Writing Guide. The sixth edition of the guide provides suggestions for appropriately including patient characteristics within case vignettes in test items. You can download the guide and find content related to patient characteristics beginning on page 46.

Test Development professionals also hosted a roundtable discussion that focused on incorporating patient characteristics in test items, including establishing terminology and managing the challenges associated with them. Some current recommendations include:

• Test items should be carefully designed to measure meaningful and plausible testing points (e.g., diagnosis, management) without the influence of assumptions, bias, or stereotypes.
• Patient characteristics should be considered based on patient self-report, not the assumption of a health care provider.
• When creating items, be mindful that race is a social construct not linked to biology or susceptibility to disease.
• Patient characteristics are sometimes necessary to significantly increase the probability of detection, diagnosis, or recognition of an otherwise rare condition.
• If no patient characteristics are given, it’s important to ensure that the item is constructed to reduce examinee personal and/or environment bias.

Teaching Social Determinants of Health

Keynote speakers Andrea Deyrup, MD, PhD, Duke University School of Medicine and Joseph Graves Jr., PhD, North Carolina Agricultural and Technical State University discussed race as a social construct in their presentation Racial Biology and Medical Misconceptions: From Curriculum to Assessment and Back Again. In addition to fostering a lively dialogue on whether patient characteristics based on race and ethnicity should be included in test items at all, the presenters stressed the importance of teaching social determinants of health within medical education.

“What’s driving these health disparities is the result of structural racism and the differential impact of poor environments and environmental conditions that predispose individuals to disease.”

JOSEPH GRAVES Jr., PhD

Engaging with the Medical Education Community

This year’s NICE spotlighted some of the challenges associated with promoting DEI in assessment and education, including mitigating bias and incorporating patient characteristics in test items. These challenges are opportunities to ensure the next generation of physicians develop in an inclusive environment and are prepared to care for a diverse patient population.

NBME looks forward to working with educators to meet this opportunity together.  

Related Content

July 2022 NBME Impact Report

NBME Continues to Support Reproductive Health and Employee Access to Care

“How can we support the future of the medical community?”

August 2021 NBME Impact Report