Personas from the Golden Collars of Software Engineers

7 personas to describe how software engineers spend their time

Advertisements

Many organizations may mistake the versatility of software engineers as universal skills they all have. However, there is no one-size fits all solution for software engineers. Basically, let’s not match software engineers with descriptions but use practical knowledge of the work they are doing to describe the many roles software engineers can take on.

To clarify how software engineers vary, we adapted Reinhardt’s knowledge worker actions to what software engineers do.

What is a knowledge worker anyway?

According to Reinhardt….

Knowledge workers have high degrees of expertise, education, or experience, and the primary purpose of their jobs involves the creation, distribution, or application of knowledge.

But for the purposes of our work let’s just simplify it to mean people who think for a living, also known as golden-collar workers. In addition, software engineering in one of the many careers that require workers to use their creative abilities to solve a problem.

Conducted Interviews + Surveys

We wanted to make sure our questions about how knowledge workers use their time was appropriately fitted for software engineers. To ensure this, we rephrased knowledge worker actions to be in the context of software development, such as the Feedback knowledge worker action now specifically referring to examples such as code review and architecture reviews.

We then interviewed 21 software engineers and received survey responses from another 868 software engineers to figure out a day in the life would look like. More specifically, we asked them what a typical day would look like for them in terms of hours, who they collaborate with, to what extent, their inputs/ outputs, and perceptions of autonomy on tasks.

Analysis

We then coded interview responses about task execution and analyzed survey responses starting with hours spent. Specifically, we performed k-means clustering on how many hours they spent performing Software Engineering knowledge worker actions in a day. Using the resulting 7 clusters, shown in the figure below, we identified significant differences between clusters based on how the software engineers described their work.

Member checking. We performed member checking by following up with interview participants about their specific time spent to determine how they align with the defined personas. We were able to group all software engineers into one of our determined personas.

clusteranalysis_larger_font_withkey.png

Interesting Findings

Collaboration Styles. In brief, we found an interesting connection between how software engineers spent their time collaborating both within and outside their teams with other roles, performing code review, and being active participants in meetings.

Personas. We reviewed our cluster and cluster descriptions and inferred 7 personas of software engineers. You can find the described persona and a brief quotes from a corresponding participant.

Acantha, the Autonomist Acumen ← Cluster 1

Acantha is known for her autonomy and keen abilities to make good judgments when writing code. On average, more than 20% of her time is spent in the Authoring knowledge worker action.

You know, I’m very much a risk-taker and I’d much rather ask for forgiveness. I’d much rather cross the line and wait for somebody to catch me. I’m willing to cross the line, and I do, and occasionally I get my hand slapped. I’m in this wonderful position.

Lilo, the Continuous Learner ← Cluster 2

Lilo is known for her interest in learning and getting adapted to a new team early in her career. She spends, on average, more than 23% of her time in the Learning knowledge worker action.

I mean it’s usually scoped but we don’t know the A, B, C’s of it, like where to start, how to start and any such things. But it usually starts with talking to different people. We know at least one contact who would then redirect us to multiple other contacts or they redirect us to some other people.

Isabelle, the Investigator ← Cluster 3

Isabelle is best known for being the top investigator on her team. She spends the most time, over 30% on average, in the Analyzing knowledge worker action.

It depends on what kind of task. For example, to fix bugs in theory you need to reproduce the problem and to just do a debug step-by-step, going into the very developed source code to figure it out. That takes a long, long time sometimes.

Cameron, the Communicator ← Cluster 4

Cameron is considered to be a communicator because of her ability to distribute her time evenly and early in her career while securing the logistics of her work environment. She is able to spread her time across all knowledge worker actions well.

The concentrated work time usually involves coding. Right now it involves learning, collating documentation, arranging resources for the people who are gonna come after me.

Iman, the Interactive ← Cluster 5

Iman is known for her activity with collaborative code writing style but also for her dynamic level of interaction that goes on within her team in terms of the Co-authoring knowledge worker action.

So we do all the code review ourselves – that’s a very important part. [Another] is the stand-ups, which have to be done – not every day, but twice in a week, we have meetings. And that’s very important because as we are still in a preview tool, so we have a lot of discussions

Ava, the Advisor ← Cluster 6

Ava is known for the high value she places on being helpful to others and giving feedback. On average, she spends over 20% of her time offering feedback.

That’s where people have an actual coding problem and they don’t know what to do to proceed, and you’re trying to give them advice or “here’s an example of code you can follow,” or “go read this article.” You’re kind of trying to hand them enough information so they can go solve the problem.

Ciara, the Team Coder ← Cluster 7

Ciara embodies this idea of being a team coder for her immense amount of time in the Co-authoring knowledge worker action (sometimes over 40%). She spends a significant amount of time producing code for her large company with a team by her side.

My role, in particular has three things that we focus on. One is basically our automated test runs. This helps us keep the business running. Basically, when the service team deploys changes or updates stuff or new content comes in, we have a suite of tests that run four times a day. So a lot of times something might roll out and because they’re the service team, they don’t really do as much robust testing from the client side.

What do we gain from this?

Inclusion through persona use. With our determined personas, software practitioners and researchers now have a resource to reference when encouraging a broader thinking of who users are. This can in turn encourage practitioners to be more inclusive of the many types of users.

Transient Modes. Software engineers may in fact transition through personas. We now have support for how to document that transition through hours spent. In the future, it may be interesting to conduct a longitudinal study that captures these transient modes based on types of projects developers work on.

Interested in the details?

I did this work as part of a summer internship in 2016! The research paper, Characterizing Software Engineering Work with Personas based on Knowledge Worker Actions by Denae Ford,Thomas Zimmermann, Christian Bird, and Nachiappan Nagappan was recently accepted at ESEM, ACM/IEEE 11th International Symposium on Empirical Software Engineering and Measurement. This year’s symposium will take place in Toronto, Canada on November 9-10, 2017. A preprint of the paper is now available online.

As always, comment(or email us) to let us know what you think about the work!

The Tech-Talk Balance: What Technical Interviewers Expect from Technical Candidates

Ever wonder what the top companies expect from their software engineering applicants?

The increased demand for software engineers in the upcoming years and their impressive salary makes the job a popular one that is highly coveted among those in STEM. With so many resources such as interviewing.io, codassium, and countless blogs about how to prepare you would think that most software engineers would have the interview process nailed down, right? WRONG!

In fact, software engineer job candidates are not exactly succeeding at technical interviews according to interviewers. Although candidates are able to answer technical questions, there is a mismatch of what candidates think interviewers assess versus what criteria is actually used in practice. Unfortunately, this mismatch in expectations can cost candidates the job of their dreams. To connect these engineers to the job of their dreams and confirm what type of mismatches may exist we sought to find out what their interviewers are looking for.

Ultimately, we were curious if there are differences across companies for how software engineer candidates are evaluated. But we were also interested in how interviewers interpret criteria for software engineer job candidates.

To soothe our curiosity, as well as other job-seeking developers, we studied interviewers in their natural habitat.

How do we really know what interviewers are expecting?

Short answer is, they told us. With interviewers from 9 major software companies, we conducted 70 mock technical interviews with software engineer candidates at our local university to better understand what companies expect from them. We anonymize the names of the company, but we give you details on their industry in the table below.

Statistical and Qualitative Analysis of Evaluations. We also collected interviewer evaluations of candidates in order to understand the expectations of interviewers and clarify how candidates should prepare for a technical interview.

After the interviews, we analyzed interviewer expectations from their evaluations and outlined how candidates can prepare for future technical interviews beyond being technically sound. We compared the Likert scores (1 through 4) interviewers gave to candidates using a Fisher’s exact test across companies followed by a Post-hoc Steel Dwass analysis on statistically significant results (α< 0:05).

What did we find in our analysis?

Evaluation Scores. In our statistical analysis, we found that all post-hoc pairs identified C1_WEB, a large internet Search company, as being significantly different than any of the other 8 companies. We did not identify any other pairs of companies with significant differences in scores for each of the evaluation criteria. We show the correspondence analysis of company ratings of candidates in the chart below. The most outstanding difference shown in this diagram demonstrates how C1_WEB scored how well candidate gave clear and concrete examples.

Criteria Interpretation. We were also able to determine how interviewers interpreted criteria used to evaluate candidates.

Example: [Original Criteria]-> [Interpretation from Interviewers]

Problem Solving-> Algorithms: When hiring candidates for a job, the top concern is whether candidates have sufficient technical skills to handle problem solving.

Nonverbal-> Interest: Interviewers noticed when there was poor communication during the interview.

Oral/Verbal Clarity->Fluent Speech: When making a first impression, the first words a candidate speaks are often the most important.

Clear, Concrete Examples-> Connected Experiences: Another way of demonstrating a candidate’s fit is their ability to communicate clear and concrete examples.

Enthusiasm-> Visible Excitement: How a candidate displayed enthusiasm is one measure of interest and engagement in the interview.

What should you share about this work?

Interviewers care about technical soundness and the ability for candidates to communicate that skill. Surprisingly, technical interviewers place an emphasis on interpersonal skills and effective communication in the interview. Interviewers wanted to hire a person, not just a candidate who can solve problems.

Most companies have consistent expectations for candidates across industry and size. According to interviewers, candidates are prepared technically, but encounter challenges translating their technical knowledge. Interviewers identified that candidates were not able make connections between previous work experiences.

Interviewers notice candidates who made the investment to prepare. It is important that candidates come prepared for a company-specific interview. We found that some companies emphasized specific technical skills.

Interested in the details?

The formal research paper,The Tech-Talk Balance: What Technical Interviewers expect from Technical Candidates by Denae Ford, Titus Barik, Leslie Rand-Pickett, and Chris Parnin was recently accepted at CHASE, IEEE 10th International Workshop on Cooperative and Human Aspects of Software Engineering. This year’s workshop is co-located with ICSE and will take place in Buenos Aires, Argentina on May 23, 2017. A pre-print of the paper is now available online.

Paradise Unplugged: Barriers to Stack Overflow Use for Females

Only 5.8% of users identified as females. We identified barriers for female contributions on Stack Overflow using semi structured interviews and surveys to validate these barriers. In this approach, we also found barriers that affected the entire community. As the big picture here is making sure all users feel comfortable using this resource, we also supplement these findings with an outline of potential site design modifications.

What is Stack Overflow and why do I care?

Stack Overflow is a popular question and answer site that many programmers go to for quick solutions to problems they encounter. Many have referred to it as “a programmer’s paradise”. Though this site is considered “a heaven sent” it is far from a true utopia. There has been much criticism of the lack of female participation on the site. In Stack Overflow’s 2015 and 2016 developer survey only 5.8% of users identified as females. The sad part about this is how often the conversation about this disparity often gets shut down with negative votes and closed as off topic discussions by moderators.

Stack Overflow does acknowledge the disappointing number saying:

Software development has a gender balance problem. Our internal stats 
suggest the imbalance isn't quite as severe as the survey results would
make it seem, but there's no doubt everyone who codes needs to be more 
proactive welcoming women into the field.

There are many movements to get women into programming, but what about keeping them there? If they don’t feel comfortable using the resources that are available for all programmers then that is a big problem for retention in the field. To do our part in being more proactive in welcoming women into the field, we sought to uncover some reasons for this low participation.

What did we do? 

We identified barriers for female contributions on Stack Overflow using semi-structured interviews and surveys to validate these barriers. In this approach, we also found barriers that affected the entire community. As the big picture here is making sure all users feel comfortable using this resource, we also supplement these findings with an outline of potential site design modifications.

How did we do it?

Interviews. In order to find out to figure out what is hindering the community we went straight to the users. We interviewed 22 female users and asked them about their experiences when they posted a comment, edited and times when their activity was not as high. Other questions asked included how people communicate on the site, personal incentives, potential scenarios that may occur when using the site and potential modifications to the site that could increase usage. From our semi-structured interviews we transcribed audio recordings, performed a card sort, and 14 distinct barriers emerged. The 14 barriers are described in the table below.

barriers_table

General Survey. After identifying the barriers we wanted to find out which barriers varied across genders. We distributed the survey to the general developer population. We sent targeted emails, posted to programming forums, contacted large corporations, and posted in computer science Facebook groups. We received responses from 1470 females and males by the time we closed the survey.

With this survey we were able to identify 5 statistically significant barriers that females rated higher than males. These barriers include being aware of site features, not feeling qualified enough to chime in, the large intimidating community size, the discomfort of online strangers, and the perception of slacking on the job. All statistically significant barriers have been starred below in the diverging stacked bar chart of all barriers identified.

OpenRating_ranked_starred
Barriers to Stack Overflow usage for females and males. Red star indicates a statistically significant difference (p<.0012)

What Now?

As mentioned before, the ultimate goal is to make sure all users can feel comfortable relying on this resource. Using the results from our interviews and survey, we propose some design choices to consider as Stack Overflow grows to be more popular and inclusive.

5 Min V.S. 30 Min Questions. It is a bit of a task trying to identify which questions will take 5 minutes to answer versus 30 minutes manually. Research is needed to create and sustain a ranking algorithm of questions response time per user’s skill and question difficulty. This features would encourage a wider range of users with varied availability.

Quality Questions. Instead of discouraging users from posting questions, enhance the posting process by automatically providing feedback on the quality of the question in terms of how fast and how likely it will be answered.

Sub-communities. Sub-communities should be an available option to users who are interested in creating a more approachable group to interact with.

Mentorship. Retention of one-day flies and other contributors may increase if  a per user mentorship program is incorporated.

Many of these design choices have been shared with Stack Overflow affiliates and are now in the implementation phase. Keep your eyes peeled to see what Stack Overflow has up their sleeves in the future.

Read More

The formal research paper, Paradise Unplugged: Identifying Barriers for Female Participation on Stack Overflow by Denae Ford, Justin Smith, Philip Guo, and Chris Parnin was recently accepted at FSE, ACM SigSoft International Symposium on the Foundations of Software Engineering. This year’s conference will be taking place in Seattle, Washington from November 13 -19, 2016. A version of the paper is now available online.

Just plain curious about the work? We welcome you to check out the paper,message us with inquiries, or comment below.

 

The FUTURE of Technical Phone Interviews

THE Problem.

For a software developer technical interviews can often feel like the most annoying process ever.  Candidates apply for a job and are called back for an interview where they are asked random intricate details about programming that are not necessary for the job. This process often feels like an intense formality, especially if you are an awesome programmer. But really, what is the gold standard for technical interviews? There are many kinds of technical interviews that candidates go through. Some of them are hackathon style (where you get into groups of 4 with strangers and code all day), riddles (where you’re asked to solve a brain teaser with pseudocode), or whiteboard coding (where you’re asked to go up to the whiteboard and write runnable code with no IDE). What can often seem overwhelming is when you are working on coding question given to you and you are instructed to ‘think-aloud’ as you work. However, everyone doesn’t work well this way. This is a skill in itself that has to be worked on strenuously especially if the interviewer is judging you based on this skill. In addition to all this, the anxiety you as the candidate experience knowing that the interviewer is studying your every word can definitely throw you off your game.

In fact, it’s the worst when you’re trying to walk through the coding example and you get interrupted by the interviewer during your thought process. This interruption at the wrong time can often result in losing your train of thought. Often times these interruptions happen on the phone when the interviewer cannot see you or cannot see the code  you are writing to solve the programming problem.

What if I told you we had the answer to your problems and I can answer them all with eye tracking?

THE Answer!

Remote Focus Lights. When driving in a car, passengers are better able to sense when a driver is busy than someone having a phone conversation with the driver. In remote interviews, a candidate may be in a mental state of high cognitive load but is not easily observable by an interviewer. For example, when writing code, an interviewer may use typing as a cue not to interrupt. But if a candidate is reflecting on a problem or reading code while deep in thought, that information is not easily accessible when on the phone. The remote focus lighjob-interviewt is an intervention that indicates when a candidate is currently involved with a high mental workload (red-circle-hi | red light) or is accessible for questions (alex-green-circle-hi |green light).

 

Remote Blackouts. A good interviewer might allow some time for a candidate to reflect on a problem in isolation, without worrying about the presence of an interviewer pressuring the candidate. For example, the interviewer might say, “now that I’ve explained the problem, I’ll put the phone down and walk out for about 4 minutes to allow you to digest the problem.” For coding activities, having your live state exposed to the interviewer can cause constant anxiety about making mistakes in front of others. A proposed intervention is the capture of the benefits of a “walk-out” during remote interviews by only refreshing the candidate’s screen in 2 minute intervals. This allows the candidate to have moments to reflect and perform simple tasks in a time-boxed manner, without constant fear of interruption.

BUT How?

I’m glad you asked. By studying task-evoked pupillary response, we can determine variations in cognitive load. Large pupil dilation is associated with high cognitive load and small pupil dilation is associated with low cognitive load. Obviously, it’s not ideal to interrupt a person when they have a high cognitive load as they are attempting to make sense of a lot of information. Using eye tracking tools we can detect when that moment occurs. The remote focus light intervention will reflect the variations in pupil dilation through a red or green light.

The nerves may feel inevitable since you unconsciously know that the interviewer is constantly rating your performance. However, you may be able to perform better if the interviewer cannot use the nonverbal cues that show your nerves. A few of these nonverbal cues include your rate of blinks, how you blink, or even where you look unconsciously through saccades. The remote blackouts provides control over your visibility to the interviewer. This control can also give you, as the candidate, confidence to continue the interview without letting your nerves show.

Future Experiment

To put these interventions to the test, we’ll recreate a technical interview environment and have candidates go through combinations of each. Further work would be to add another dynamic of this study with the different types of interview styles mentioned earlier. We can then study which types of interviews work well for different types of programmers and develop a true gold standard for technical interviews. The reduced cost in eye-tracking tools make this functionality now more available than ever and technical interviews provide a realistic platform for studies of this nature to be conducted.

Want to Know More?

The paper, Studying Sustained Attention and Cognitive States in Remote Technical Interviews, (Authored by myself, Titus Barik and Chris Parnin) has been accepted for presentation at the 2015 International Workshop Eye Movements in Programming Workshop, in Joensuu, Finland. I presented this work on November 2015 but will still be responding actively to inquiries about the project on twitter!

 

Images source: http://www.nadinemuller.org.uk/wp-content/uploads/2013/03/job-interview.jpg

Exploring Causes of Frustration for Software Developers

The lifestyle of a developer includes constantly learn new technologies, adapting to new environments, and overcoming challenges when learning and practicing their craft. Unfortunately, failure to overcome obstacles in these situations can introduce a sense of mounting frustration in developers that can negatively impact learning outcomes and influence retention in a field. Aside from heated venting session on Reddit, Twitter, and other blogging sites there has been a lack of a concerted effort in cataloging and categorizing the frustrating experiences developers face. Understanding these frustrating experiences is the first step to alleviating them. To gain better grasp, we sent a survey to software developers asking them to describe their frustrating experiences.  From the survey, 67% of respondents acknowledged that frustration is a severe problem for them. With the 45 responses we received, we were able to define a taxonomy of the causes.

P.I.L.E. of Trouble

These frustrations were grouped into 4 categories Perception- how the individual views the issue, Internal- sprouting from some internal conflict, Lack Of Experience- situations that require generally more familiarity which comes with time, and External- sprouting from something out of the individual’s control.

Under the hood of these categories the groups of frustration were defined  as follows:

Perception

  • Size:
    • (1) large goal spaces, high cognitive complexity: “Where do I begin? “
    • (2) gulf of completion: “I’m almost done but there is a little thing in my way”
    • (3) large artifacts: “There’s so much to understand”

Internal

  • Fear of Failure: The obsession over the fear of not succeeding setting the individual back
  • Internal Hurdles: Individuals placing pressure on themselves
  • Simple Problem: Management and peers downplaying the intensity of tasks

Lack Of Experience

  • Mapping Behavior to Cause: Difficulties identifying the connections between portions of code; having issues creating a mental model
  • Programming Experience: Learning a new language warrants learning a new syntax
  • New Project Adjustment: Getting acclimated to a new project
  • Programming Tools : The learning curve of new tools includes learning new features and shortcuts; broken tools add to the burden

External

  • Unavailability of Resources: Undocumented code creates confusion
  • Peers: Sometimes it’s hard to resist the urge to help your neighbor, what about when your neighbor needs too much from you?
  • Limited Time: Not enough time allotted to complete a task

Multipliers

In comparing the causes, we found instances where one causes contributed to another. We called these instances multipliers. Multipliers acknowledged what supplemental frustrations could have increased the frustration of the participant. However, every response does not have a multiplier. Multipliers were not mentioned in the single response of every participant but were conceived from further questions asked during the survey. This discovery opened the opportunity to explore this taxonomy further.

Next Steps

After this study we found that a deep understanding of frustration in software development is a field that has a lot of questions that remain unanswered. There are a couple questions that we plan on exploring in the future:

  • How often do these causes show up?
  • Are any of them more severe than others?
  • Do these situations vary across genders?
  • How do certain multipliers affect other causes?

Read More

The paper, Exploring Causes of Frustration for Software Developers, presenting these results (co-authored by Denae Ford and Chris Parnin) has been accepted for presentation at the 2015 ICSE Cooperative and Human Aspects of Software Engineering Workshop, in Florence, Italy, in May 2015. In addition, a buzzfeed post has been published with these findings.

11 Signs of A Frustrated Programmer

Check out this BuzzFeed Post I created about my research. It’s titled 11 Signs of a Frustrated Programmer. Read it and let me know what you think.

frustrated