THE Problem.
For a software developer technical interviews can often feel like the most annoying process ever. Candidates apply for a job and are called back for an interview where they are asked random intricate details about programming that are not necessary for the job. This process often feels like an intense formality, especially if you are an awesome programmer. But really, what is the gold standard for technical interviews? There are many kinds of technical interviews that candidates go through. Some of them are hackathon style (where you get into groups of 4 with strangers and code all day), riddles (where you’re asked to solve a brain teaser with pseudocode), or whiteboard coding (where you’re asked to go up to the whiteboard and write runnable code with no IDE). What can often seem overwhelming is when you are working on coding question given to you and you are instructed to ‘think-aloud’ as you work. However, everyone doesn’t work well this way. This is a skill in itself that has to be worked on strenuously especially if the interviewer is judging you based on this skill. In addition to all this, the anxiety you as the candidate experience knowing that the interviewer is studying your every word can definitely throw you off your game.
In fact, it’s the worst when you’re trying to walk through the coding example and you get interrupted by the interviewer during your thought process. This interruption at the wrong time can often result in losing your train of thought. Often times these interruptions happen on the phone when the interviewer cannot see you or cannot see the code you are writing to solve the programming problem.
What if I told you we had the answer to your problems and I can answer them all with eye tracking?
THE Answer!
Remote Focus Lights. When driving in a car, passengers are better able to sense when a driver is busy than someone having a phone conversation with the driver. In remote interviews, a candidate may be in a mental state of high cognitive load but is not easily observable by an interviewer. For example, when writing code, an interviewer may use typing as a cue not to interrupt. But if a candidate is reflecting on a problem or reading code while deep in thought, that information is not easily accessible when on the phone. The remote focus ligh
t is an intervention that indicates when a candidate is currently involved with a high mental workload (
| red light) or is accessible for questions (
|green light).
Remote Blackouts. A good interviewer might allow some time for a candidate to reflect on a problem in isolation, without worrying about the presence of an interviewer pressuring the candidate. For example, the interviewer might say, “now that I’ve explained the problem, I’ll put the phone down and walk out for about 4 minutes to allow you to digest the problem.” For coding activities, having your live state exposed to the interviewer can cause constant anxiety about making mistakes in front of others. A proposed intervention is the capture of the benefits of a “walk-out” during remote interviews by only refreshing the candidate’s screen in 2 minute intervals. This allows the candidate to have moments to reflect and perform simple tasks in a time-boxed manner, without constant fear of interruption.
BUT How?
I’m glad you asked. By studying task-evoked pupillary response, we can determine variations in cognitive load. Large pupil dilation is associated with high cognitive load and small pupil dilation is associated with low cognitive load. Obviously, it’s not ideal to interrupt a person when they have a high cognitive load as they are attempting to make sense of a lot of information. Using eye tracking tools we can detect when that moment occurs. The remote focus light intervention will reflect the variations in pupil dilation through a red or green light.
The nerves may feel inevitable since you unconsciously know that the interviewer is constantly rating your performance. However, you may be able to perform better if the interviewer cannot use the nonverbal cues that show your nerves. A few of these nonverbal cues include your rate of blinks, how you blink, or even where you look unconsciously through saccades. The remote blackouts provides control over your visibility to the interviewer. This control can also give you, as the candidate, confidence to continue the interview without letting your nerves show.
Future Experiment
To put these interventions to the test, we’ll recreate a technical interview environment and have candidates go through combinations of each. Further work would be to add another dynamic of this study with the different types of interview styles mentioned earlier. We can then study which types of interviews work well for different types of programmers and develop a true gold standard for technical interviews. The reduced cost in eye-tracking tools make this functionality now more available than ever and technical interviews provide a realistic platform for studies of this nature to be conducted.
Want to Know More?
The paper, Studying Sustained Attention and Cognitive States in Remote Technical Interviews, (Authored by myself, Titus Barik and Chris Parnin) has been accepted for presentation at the 2015 International Workshop Eye Movements in Programming Workshop, in Joensuu, Finland. I presented this work on November 2015 but will still be responding actively to inquiries about the project on twitter!
Images source: http://www.nadinemuller.org.uk/wp-content/uploads/2013/03/job-interview.jpg
fordable 
