The Diversity-Innovation Paradox in Science

Researchers at Stanford University were already familiar with something called the “diversity paradox.”

The diversity paradox is the hypothesis that people from diverse backgrounds and perspectives can hasten innovation because “historically underrepresented groups often draw relations between ideas and concepts that have been traditionally missed or ignored,” the researchers recently noted in their study “The Diversity-Innovation Paradox in Science.”

The “paradox” is that although these groups contribute accelerated innovation in organizations, they also typically “have less successful careers within them.”

With this in mind, the team from Stanford’s Graduate School of Education, Department of Computer Science and Department of Linguistics wondered if the diversity paradox also applies in the sciences.

The question inspired a study focused on a dataset of 1,208,246 dissertations (collected within ProQuest Dissertation & These Global) filed by US doctorate-awarding universities from 1977 to 2015. The goal of the study was to understand “if gender and racially underrepresented scholars are likely to innovate and innovation supposedly leads to successful academic careers, then how do we explain persistent inequalities in scientific careers between minority and majority groups?”

Using text analysis and machine learning (via ProQuest TDM Studio), the research team explored three key areas:

• How do we detect scientific innovations?
• Are underrepresented groups more likely to generate scientific innovations?
• Are the innovations of underrepresented groups adopted and rewarded?

“Our analyses show that underrepresented groups produce higher rates of scientific novelty,” the Stanford team concluded. “However, their novel contributions are devalued and discounted.”

“For example,” researchers explained:

[N]ovel contributions by gender and racial minorities are taken up by other scholars at lower rates than novel contributions by gender and racial majorities, and equally impactful contributions of gender and racial minorities are less likely to result in successful scientific careers than for majority groups.

“These results suggest there may be unwarranted reproduction of stratification in academic careers that discounts diversity’s role in innovation and partly explains the underrepresentation of some groups in academia,” according to their published paper.

Read the paper in its entirety, published in 2020 by PNAS (Proceedings of the National Academy of Sciences of the United States of America), or learn about how researchers at Stanford and other universities are mining ProQuest dissertations using the TDM Studio analysis tools to uncover patterns and trends in advanced scholarship.

This research was a collaborative effort between the Graduate School of Education, Department of Computer Science, and Department of Linguistics at Stanford University. Dutch researcher, Dr. B. Hofstra, Assistant Professor of Radboud University, The Netherlands worked alongside Daniel McFarland, Professor of Education and Sociology, Stanford University. The paper was supported by three NSF grants and one grant from the Dutch Organization for Scientific Research (NOW Grant).