Assessing and interviewing data engineers from a distance
When in-person technical interviews are no longer an option, hiring managers still have a wealth of online resources at their disposal. With a properly designed interview process, teams can effectively and efficiently assess whether an applicant has the right set of data engineering skills without having to physically be in the same room.
Since 2014, Insight been successfully running a fully distributed and fully remote interviewing process that has helped us sift through thousands of applications and identify top-tier candidates who have joined our Fellowship programs and gone on to work as data engineers at Netflix, Facebook, Vanguard, Apple, Bosch, and others. We recently shared some of our best practices during a live webinar (you can watch the video here). These are some of our main takeaways:
- Standardize your process to ensure a consistent candidate experience.
- Set clear expectations for candidates and communicate upfront as much as possible.
- Hone in on the skills that are crucial to who you are seeking to hire.
- Design assessments that will highlight applicants’ desired skills in a way that is achievable.
- Engage in dialogue with applicants and involve other team members in the process.
Cassie Stover, Insight’s Director of Admissions, has spent several years fine-tuning a large-scale application and interviewing process that allows her and the rest of the Insight team to review more than 12,000 applications and conduct upwards of 3,700 technical interviews annually.
With such a high volume of interviews, standardizing the interview process helps keep everyone organized and provides a consistent candidate experience to maintain a level of fairness and integrity in the process. Stover recommends streamlining the many different touch points in the process, for instance, creating email templates so that all applicants receive the same amount of information in the same manner.
When there are many candidates going through the interview process with multiple steps at each stage, it’s important to pick tools that integrate with one another automatically. For instance, having tools that automatically integrate interview scheduling (e.g., ClaraAI) with a company’s calendaring system (e.g., Google Calendar), remote video platform (e.g., Zoom), and applicant tracking system (e.g., Greenhouse) will help ease the interview process.
Consistently communicating with your applicants on what they can expect at each stage of the process, such as setting the tone, the format of the interview, and the timeline, is also important. In some cases, it’s better to over-communicate expectations with applicants because it puts them at ease and allows them to feel better prepared.
As Director of Engineering at Insight, Hoa Nguyen has personally interviewed and evaluated hundreds of applicants for our Fellows programs. She’s also worked closely with many companies to design and improve their interviewing processes to find the best possible engineering talent for their teams.
Before starting the process of evaluating your applicants, it’s important to identify who you are hiring and what skills you’d like to have. That information is key to creating and tailoring your technical assessments toward those skills.
Having run a data engineering program at Insight for several years, we’ve identified three broad categories of data engineers:
- Software engineers who focus on building data pipelines. In some cases, they work to deploy data science models into production with an eye towards optimization, scalability and maintainability.
- Infrastructure engineers who are tasked with making sure the backend systems are up and running, including creating and maintaining APIs and micro-services.
- Data architects and data modelers who specialize in areas such as schema design, identifying query access patterns and building and maintaining data warehouses.
Based on the work data engineers do, they must have certain technical skills, such as the ability to code and work on backend software development, interest and experience working with diverse and growing data stores, as well as the knowledge and ability to deal with engineering challenges around scalability, optimization, resilience and maintainability. And finally, because data engineers often work with different user groups, they must be communicative and collaborative.
At Insight, our Data Engineering Fellows program is a collaborative learning space for people with strong computer science fundamentals who want to transition to data engineering through hands-on experience with distributed computing technologies. That means Fellows must be able to code and put together a project using version control in a Linux environment. They must also have the potential to grow and benefit from the Insight Fellowship experience, and finally fit well into our collaborative and congenial environment.
Our interview process at Insight consists of a coding assessment, application review, technical video interview and any necessary follow-ups.
Coding Assessments
When it comes to coding assessments, there are may different types widely used in industry. One of the advantages of using a coding assessment is the evidence they can provide regarding candidates’ technical abilities. Also, some candidates may be unable to share past projects they have worked on due to non-disclosure agreements. Having an independent technical assessment that allows candidates to showcase their skills can help. Providing a coding assessment also allows the interviewer and the candidate to be on the same page during any subsequent interviews, so they can discuss the work in more depth, and dig a little deeper into some of the technical details.
Some of the disadvantages of using a coding assessment are that it does take time and, in some cases, if you use an off-the-shelf assessment tool (such as HackerRank) it will cost money. Another major disadvantage with using an assessment tool is that if you use the same one for multiple candidates, there’s a high likelihood it will have been shared on the internet (e.g. these are common on Glassdoor) so there is a possibility that cheating may occur.
While many companies use coding assessments with computer science algorithm questions (commonly found on platforms such as LeetCode and HackerRank), other companies may opt for a more open-ended take-home problem. For example, they may give applicants access to an API and ask them to query data that satisfies some criteria, or they may share a large dataset and asking applicants to perform some sort of data transformation.
At Insight, we use a hybrid approach where we give applicants a link to a problem statement detailed on a Github. The problem requires use of one or two foundational data structures and details some sort of analysis that we’d like performed on a dataset. Each submission is run through a series of tests to ensure that the desired output is produced. Each submission is also manually reviewed by a program director who is looking for code quality and software best practices.
Application Review
Coding assessments are reviewed first — even before the application is read. That allows us to reduce bias and fairly evaluate the coding assessment. The results of the coding assessment can help overcome gaps in a candidate’s technical background, while a poorly executed assessment can temper the experience an applicant lists on a resume.
After the coding assessment, we review a candidate’s application in advance of a technical interview so that we can be prepared for the areas we want to dig into during the technical video interview. The agenda for our video interviews are high-level and open-ended, primarily consisting of introductions, walk-through of the coding assessment, and then a conversation with the applicant.
Technical Video Interview
Reviewing the coding assessment during a live video interview is generally about ensuring that the applicant wrote the code and put some thought into how they solved the problem. We will also use the coding assessment as a starting point in assessing their interest in system design and working with data as challenges such as scalability become a factor.
After the coding assessment, we use the rest of the time during the video interview chatting with applicants about their career aspirations and what they hope to get out of Insight.
Following the video interview, we put together our notes on how the interview went. Most importantly, if we ran out of time and there were questions that were left unanswered, or there was more to discuss, it’s important to note all of that information. At that point, we will sometimes recommend that a second colleague conduct a follow-up interview to address any remaining questions.
For additional resources to help you develop your own remote interview process, check out our video as well as our Guide to Conducting Remote Interviews. In this guide, we’ve compiled advice based on Insight’s own experience operating a 100% remote interview process for our Fellows programs, as well as best practices amassed from working with hundreds of hiring teams, ranging from small startups to Fortune 500 enterprises. These tips will help you build a remote interviewing process that fits your needs and enables you to make sound decisions as you grow your team.
Thinking about adapting your existing processing to remote? Sign up for our monthly newsletter or get in touch with our team to learn more.
