Men were 1.4 times as likely as women to advance to the next round on the website and on average scored 3 out of 5 on technical ability, while women scored 2.5.
"I was surprised," she said.
"To me, there is nothing about programming a computer that ought to favor one gender over another. And technical interviews, especially, are a learned skill."
The data inspired her to adopt voice-masking technology as part of the interview process. The tool modulates the candidate's voice in real time, either to the opposite gender's voice or to a gender-neutral one. The tool also preserves the candidate's vocal inflections so that the person does not come across as a robot.
When Lerner began the experiment, she asked participants on her website whether they wanted to opt in to using voice-masking software, but she did not tell them their voices would be modulated to sound like the opposite gender.
Interviewers were informed that participants' voices might sound slightly processed. Candidates were asked not to mention their genders throughout the interviews. Sixty-three participants opted in for a total of 234 interviews.
The result?
The men whose voices were modulated to sound feminine were rated slightly higher than those whose were not. On the flip side, women's voices that were modulated to sound masculine were rated a bit lower than when unmodulated.
To me, there is nothing about programming a computer that ought to favor one gender over another. And technical interviews, especially, are a learned skill.
Lerner stresses that the trend was not enough to be statistically significant but does point toward something surprising.
Why, given that result, were women on average performing worse?
The answer may lie in another data trend that Lerner noticed: rate of attrition.
After an interview was rated poorly by the interviewer, women were seven times as likely to leave and never return to the site as men who also received poor ratings. After the second interview, poorly rated women continued to leave at a higher rate than poorly rated men.
The data suggests that the more people returned to the site to try again, the more their performances improved. Lerner tested this: She threw out the users who quit after a bad interview and found that the performance gap between men and women closed.
"In our experience, after two or three interviews, people tend to start finding their groove. Like any other learned skill, practice makes you better," Lerner said.
But she stresses the limitations of this experiment, including a small sample size and the limitations of using voice-swapping alone to communicate one's gender.
In our experience, after two or three interviews, people tend to start finding their groove. Like any other learned skill, practice makes you better.
But the experiment's conclusions do fall in line with other studies that examine possible reasons for the gap between women and men in tech.
In a Cornell University study, researchers tested scientific reasoning skills in women and men. They found that there was no difference in ability but that women repeatedly underrated their own performances. And in a study published by the independent organization Sage Publishing, researchers examined diary entries of college students in science, technology, engineering and math studies.
They found that when female students encountered evidence that they were not performing at the top of their classes, "the experience [triggered] a more fundamental doubt about their abilities to master the technical constructs of engineering expertise" than it did with men.
This issue is not confined to the tech space.
Kevin Miller, senior researcher at the American Association of University Women, agrees that the confidence gap is an important factor in gender disparities within the workforce.
I do hope that we can start a conversation around the gender gap through the lens of grit . . . rather than just the narrative that the tech industry is biased against women as a whole.
"Men are more likely to apply for jobs in the first place, even if they aren't fully qualified, whereas women will assume that they need to be fully qualified in order to apply," he said, citing a Harvard Business Review study.
He also noted that one reason for the gender gap in political office is that women do not run as much.
Still, Miller said, using tools such as voice-masking software to make the interview process anonymous can help remove potential unconscious bias on the hiring end. He said the Toronto Symphony Orchestra introduced blind recruitment in the 1980s by auditioning musicians behind a screen.
The result was a much more racially diverse orchestra with a roughly even split of male and female players.
Blind recruitment has been introduced in many companies in recent years.
Deloitte, HSBC and the BBC are among those that have chosen to recruit employees based on résumés stripped of names and schools. The hope is to remove possible bias, promote social mobility and introduce employee diversity that could better reflect an increasingly diverse customer base.
For Lerner, her platform is just one step toward a more meritocratic and diverse playing field.
"I do hope that we can start a conversation around the gender gap through the lens of grit . . . rather than just the narrative that the tech industry is biased against women as a whole," she said.
"That's not to say that biases don't exist or that the playing field is level. I just think that this problem is going to need a lot of different approaches to fix."
