The increased demand for software engineers in the upcoming years and their impressive salary makes the job a popular one that is highly coveted among those in STEM. With so many resources such as interviewing.io, codassium, and countless blogs about how to prepare you would think that most software engineers would have the interview process nailed down, right? WRONG!
In fact, software engineer job candidates are not exactly succeeding at technical interviews according to interviewers. Although candidates are able to answer technical questions, there is a mismatch of what candidates think interviewers assess versus what criteria is actually used in practice. Unfortunately, this mismatch in expectations can cost candidates the job of their dreams. To connect these engineers to the job of their dreams and confirm what type of mismatches may exist we sought to find out what their interviewers are looking for.
Ultimately, we were curious if there are differences across companies for how software engineer candidates are evaluated. But we were also interested in how interviewers interpret criteria for software engineer job candidates.
To soothe our curiosity, as well as other job-seeking developers, we studied interviewers in their natural habitat.
How do we really know what interviewers are expecting?
Short answer is, they told us. With interviewers from 9 major software companies, we conducted 70 mock technical interviews with software engineer candidates at our local university to better understand what companies expect from them. We anonymize the names of the company, but we give you details on their industry in the table below.
Statistical and Qualitative Analysis of Evaluations. We also collected interviewer evaluations of candidates in order to understand the expectations of interviewers and clarify how candidates should prepare for a technical interview.
After the interviews, we analyzed interviewer expectations from their evaluations and outlined how candidates can prepare for future technical interviews beyond being technically sound. We compared the Likert scores (1 through 4) interviewers gave to candidates using a Fisher’s exact test across companies followed by a Post-hoc Steel Dwass analysis on statistically significant results (α< 0:05).
What did we find in our analysis?
Evaluation Scores. In our statistical analysis, we found that all post-hoc pairs identified C1_WEB, a large internet Search company, as being significantly different than any of the other 8 companies. We did not identify any other pairs of companies with significant differences in scores for each of the evaluation criteria. We show the correspondence analysis of company ratings of candidates in the chart below. The most outstanding difference shown in this diagram demonstrates how C1_WEB scored how well candidate gave clear and concrete examples.
Criteria Interpretation. We were also able to determine how interviewers interpreted criteria used to evaluate candidates.
Example: [Original Criteria]-> [Interpretation from Interviewers]
Problem Solving-> Algorithms: When hiring candidates for a job, the top concern is whether candidates have sufficient technical skills to handle problem solving.
Nonverbal-> Interest: Interviewers noticed when there was poor communication during the interview.
Oral/Verbal Clarity->Fluent Speech: When making a first impression, the first words a candidate speaks are often the most important.
Clear, Concrete Examples-> Connected Experiences: Another way of demonstrating a candidate’s fit is their ability to communicate clear and concrete examples.
Enthusiasm-> Visible Excitement: How a candidate displayed enthusiasm is one measure of interest and engagement in the interview.
What should you share about this work?
Interviewers care about technical soundness and the ability for candidates to communicate that skill. Surprisingly, technical interviewers place an emphasis on interpersonal skills and effective communication in the interview. Interviewers wanted to hire a person, not just a candidate who can solve problems.
Most companies have consistent expectations for candidates across industry and size. According to interviewers, candidates are prepared technically, but encounter challenges translating their technical knowledge. Interviewers identified that candidates were not able make connections between previous work experiences.
Interviewers notice candidates who made the investment to prepare. It is important that candidates come prepared for a company-specific interview. We found that some companies emphasized specific technical skills.
Interested in the details?
The formal research paper,The Tech-Talk Balance: What Technical Interviewers expect from Technical Candidates by Denae Ford, Titus Barik, Leslie Rand-Pickett, and Chris Parnin was recently accepted at CHASE, IEEE 10th International Workshop on Cooperative and Human Aspects of Software Engineering. This year’s workshop is co-located with ICSE and will take place in Buenos Aires, Argentina on May 23, 2017. A pre-print of the paper is now available online.
fordable 