Free and Open Source Software

From the post's link:

We hypothesized that pull requests made by women are less likely to be accepted than those made by men. Prior work on gender bias in hiring – that women tend to have resumes less favorably evaluated than men (5) – suggests that this hypothesis may be true.

To evaluate this hypothesis, we looked at the pull status of every pull request submitted by women compared to those submitted by men. We then calculate the merge rate and corresponding confidence interval, using the Clopper-Pearson exact method (15), and find the following:

	Open	Closed	Merged	Merge Rate	95% Confidence Interval
Women	8,216	21,890	111,011	78.6%	[78.45%, 78.87%]
Men	150,248	591,785	2,181,517	74.6%	[74.56%, 74.67%]

4 percentage point difference overall.

Pull requests can be made by anyone, including both insiders (explicitly authorized owners and collaborators) and outsiders (other GitHub users). If we exclude insiders from our analysis, the women’s acceptance rate (64.4%) continues to be significantly higher than men’s (62.7%) (χ2(df = 2, n = 2, 473, 190) = 492, p < .001)

Emphasis mine. that's 1.7 percentage points.

The final paragraph also omits how the acceptance changes after gender is "revealed" (username, profile image). The graph doesn't help either

For outsiders, we see evidence for gender bias: women’s acceptance rates are 71.8% when they use gender neutral profiles, but drop to 62.5% when their gender is identifiable. There is a similar drop for men, but the effect is not as strong. Women have a higher acceptance rate of pull requests overall (as we reported earlier), but when they’re outsiders and their gender is identifiable, they have a lower acceptance rate than men.

So women drop from 71.8% to 62.5% = 9,3 percentage points, and they say it's more than men, but don't reveal the difference. Only graph has an indication (unless I'm missing a table) and it may be 5 (?) percentage points for men. Which would be about 4 percentage points between both genders.

Figure 5: Pull request acceptance rate by gender and perceived gender, with 95% Clopper-Pearson confidence intervals, for insiders (left) and outsiders (right)

The conclusion:

Our results suggest that although women on GitHub may be more competent overall, bias against them exists nonetheless.

That's quite exaggerated for <=5 percentage points. Especially for the number of people involved.

Out of 4,037,953 GitHub user profiles with email addresses, we were able to identify 1,426,121 (35.3%) of them as men or women through their public Google+ profiles.

Maybe I missed it, but how many of those were women and how many made PRs?

in a 2013 survey of the more than 2000 open source developers who indicated a gender, only 11.2% were women

Let's compare the PR rate per gender:

Let's say the percentage of women did not increase since 2013, which I'd find difficult to believe, that's 1,269,247 men and 156,873 women. Men made 150,248 + 591,785 + 2,181,517 = 2,923,550 PRs. Women made 8,216 + 21,890 + 111,011 = 141,117 PRs. That's ~2.3 PRs per man and ~0,9 PRs per woman. If the percentage changed and more women became contributors, that would decrease the PRs per woman.

That leads me to ask:

• are women more hesitant to contribute PRs that might not be merged? if so, it might contribute to why their PRs are merged more often
• are the women with accounts on github more likely to be people who have some kind of education in the IT field? if there are less hobbyist women (percentage-wise) on github, and more hobbyist men who just chuck their stuff online then decide to contribute to a project, it might contribute to PR acceptance (you're comparing pros to amateurs)
• what does a similar acceptance rate for double the amount of PRs for men actually say? I don't know, but it might be pertinent.

I very much encourage humans to contribute to opensource. So, while this paper says something about the current state of things, it doesn't seem like it's saying much. The differences in pull request acceptance are not very significant (<5 percentage points) to me

Anti Commercial-AI license