Paradise Unplugged: Barriers to Stack Overflow Use for Females

What is Stack Overflow and why do I care?

Stack Overflow is a popular question and answer site that many programmers go to for quick solutions to problems they encounter. Many have referred to it as “a programmer’s paradise”. Though this site is considered “a heaven sent” it is far from a true utopia. There has been much criticism of the lack of female participation on the site. In Stack Overflow’s 2015 and 2016 developer survey only 5.8% of users identified as females. The sad part about this is how often the conversation about this disparity often gets shut down with negative votes and closed as off topic discussions by moderators.

Stack Overflow does acknowledge the disappointing number saying:

Software development has a gender balance problem. Our internal stats 
suggest the imbalance isn't quite as severe as the survey results would
make it seem, but there's no doubt everyone who codes needs to be more 
proactive welcoming women into the field.

There are many movements to get women into programming, but what about keeping them there? If they don’t feel comfortable using the resources that are available for all programmers then that is a big problem for retention in the field. To do our part in being more proactive in welcoming women into the field, we sought to uncover some reasons for this low participation.

What did we do? 

We identified barriers for female contributions on Stack Overflow using semi-structured interviews and surveys to validate these barriers. In this approach, we also found barriers that affected the entire community. As the big picture here is making sure all users feel comfortable using this resource, we also supplement these findings with an outline of potential site design modifications.

How did we do it?

Interviews. In order to find out to figure out what is hindering the community we went straight to the users. We interviewed 22 female users and asked them about their experiences when they posted a comment, edited and times when their activity was not as high. Other questions asked included how people communicate on the site, personal incentives, potential scenarios that may occur when using the site and potential modifications to the site that could increase usage. From our semi-structured interviews we transcribed audio recordings, performed a card sort, and 14 distinct barriers emerged. The 14 barriers are described in the table below.

barriers_table

General Survey. After identifying the barriers we wanted to find out which barriers varied across genders. We distributed the survey to the general developer population. We sent targeted emails, posted to programming forums, contacted large corporations, and posted in computer science Facebook groups. We received responses from 1470 females and males by the time we closed the survey.

With this survey we were able to identify 5 statistically significant barriers that females rated higher than males. These barriers include being aware of site features, not feeling qualified enough to chime in, the large intimidating community size, the discomfort of online strangers, and the perception of slacking on the job. All statistically significant barriers have been starred below in the diverging stacked bar chart of all barriers identified.

OpenRating_ranked_starred

Barriers to Stack Overflow usage for females and males. Red star indicates a statistically significant difference (p<.0012)

What Now?

As mentioned before, the ultimate goal is to make sure all users can feel comfortable relying on this resource. Using the results from our interviews and survey, we propose some design choices to consider as Stack Overflow grows to be more popular and inclusive.

5 Min V.S. 30 Min Questions. It is a bit of a task trying to identify which questions will take 5 minutes to answer versus 30 minutes manually. Research is needed to create and sustain a ranking algorithm of questions response time per user’s skill and question difficulty. This features would encourage a wider range of users with varied availability.

Quality Questions. Instead of discouraging users from posting questions, enhance the posting process by automatically providing feedback on the quality of the question in terms of how fast and how likely it will be answered.

Sub-communities. Sub-communities should be an available option to users who are interested in creating a more approachable group to interact with.

Mentorship. Retention of one-day flies and other contributors may increase if  a per user mentorship program is incorporated.

Many of these design choices have been shared with Stack Overflow affiliates and are now in the implementation phase. Keep your eyes peeled to see what Stack Overflow has up their sleeves in the future.

Read More

The formal research paper, Paradise Unplugged: Identifying Barriers for Female Participation on Stack Overflow by Denae Ford, Justin Smith, Philip Guo, and Chris Parnin was recently accepted at FSE, ACM SigSoft International Symposium on the Foundations of Software Engineering. This year’s conference will be taking place in Seattle, Washington from November 13 -19, 2016. A version of the paper is now available online.

Just plain curious about the work? We welcome you to check out the paper,message us with inquiries, or comment below.

 

Exploring Causes of Frustration for Software Developers

The lifestyle of a developer includes constantly learn new technologies, adapting to new environments, and overcoming challenges when learning and practicing their craft. Unfortunately, failure to overcome obstacles in these situations can introduce a sense of mounting frustration in developers that can negatively impact learning outcomes and influence retention in a field. Aside from heated venting session on Reddit, Twitter, and other blogging sites there has been a lack of a concerted effort in cataloging and categorizing the frustrating experiences developers face. Understanding these frustrating experiences is the first step to alleviating them. To gain better grasp, we sent a survey to software developers asking them to describe their frustrating experiences.  From the survey, 67% of respondents acknowledged that frustration is a severe problem for them. With the 45 responses we received, we were able to define a taxonomy of the causes.

P.I.L.E. of Trouble

These frustrations were grouped into 4 categories Perception- how the individual views the issue, Internal- sprouting from some internal conflict, Lack Of Experience- situations that require generally more familiarity which comes with time, and External- sprouting from something out of the individual’s control.

Under the hood of these categories the groups of frustration were defined  as follows:

Perception

  • Size:
    • (1) large goal spaces, high cognitive complexity: “Where do I begin? “
    • (2) gulf of completion: “I’m almost done but there is a little thing in my way”
    • (3) large artifacts: “There’s so much to understand”

Internal

  • Fear of Failure: The obsession over the fear of not succeeding setting the individual back
  • Internal Hurdles: Individuals placing pressure on themselves
  • Simple Problem: Management and peers downplaying the intensity of tasks

Lack Of Experience

  • Mapping Behavior to Cause: Difficulties identifying the connections between portions of code; having issues creating a mental model
  • Programming Experience: Learning a new language warrants learning a new syntax
  • New Project Adjustment: Getting acclimated to a new project
  • Programming Tools : The learning curve of new tools includes learning new features and shortcuts; broken tools add to the burden

External

  • Unavailability of Resources: Undocumented code creates confusion
  • Peers: Sometimes it’s hard to resist the urge to help your neighbor, what about when your neighbor needs too much from you?
  • Limited Time: Not enough time allotted to complete a task

Multipliers

In comparing the causes, we found instances where one causes contributed to another. We called these instances multipliers. Multipliers acknowledged what supplemental frustrations could have increased the frustration of the participant. However, every response does not have a multiplier. Multipliers were not mentioned in the single response of every participant but were conceived from further questions asked during the survey. This discovery opened the opportunity to explore this taxonomy further.

Next Steps

After this study we found that a deep understanding of frustration in software development is a field that has a lot of questions that remain unanswered. There are a couple questions that we plan on exploring in the future:

  • How often do these causes show up?
  • Are any of them more severe than others?
  • Do these situations vary across genders?
  • How do certain multipliers affect other causes?

Read More

The paper, Exploring Causes of Frustration for Software Developers, presenting these results (co-authored by Denae Ford and Chris Parnin) has been accepted for presentation at the 2015 ICSE Cooperative and Human Aspects of Software Engineering Workshop, in Florence, Italy, in May 2015. In addition, a buzzfeed post has been published with these findings.