Marvik + Experfy | Candidate Recommendation Engine

Case Study

Candidate recommendation engine

Building a recommendation engine capable of using Natural Language Processing to automatically parse thousands of candidate resumes and recommend them to a job opportunity

Learn More

Experfy is a startup that kicked off from the Harvard Innovation Lab in Boston and Deloitte’s hiring partner for data science and artificial intelligence projects. They provide top of the notch experts for demanding projects. Experfy has a pool of specialists who can be recommended for new positions at Deloitte’s client’s.

The goal

Initially the client’s talent team could manually recommend the best candidates for any given project but this soon became a problem as the expert’s and project’s list began to grow.

We were asked to build a recommendation engine that could help find the best professionals for any given project using their skills, past experiences and education.

The Data

Although the client had rich information, it was not structured in a way that we could easily use.

For example, we had to read uploaded resumes to extract some relevant skills or past experiences. We not only had to build a recommendation engine but also a natural language processing system to understand both recruiters and experts. Business rules also had to be applied, filtering candidates by nearby locations, rates and availability.

Our Solution

Extracting and matching skills from a project and a resume is not an easy task. It might be easy for a human to understand that Mysql and Postgres are really similar skills but it’s not easy to teach that to a machine. The problem escalates when you have an ever expanding database of candidates with new skills. At this point, a rules engine is just not up to the task. another challenge was we had to make the recommendations in real time.

Our work involved combining skills embeddings trained for this specific purpose with a ranking system for which we ended up using Yandex’s Catboot. The system’s features are actually more complex than just a recommendation (we use machine learning for multiple tasks that help reduce manual work). Different requirements forced us to use more precise metrics such as word mover’s distance or wmd when we needed to be accurate, and simpler metrics when we needed to be faster. These additional features added to the system allowed us to include Google’s BERT question answering.

Client’s freelance jobs platform

We teamed up with the client’s IT team to deliver a microservice that was integrated with the existing architecture, allowing the client to scale up operations:

Technology

Technologies used involved Python, Golang and Tensorflow. All got deployed to GPU servers as speed was important. We implemented a CI&CD pipeline using Docker and Kubernetes in AWS. Automatic tests were implemented in both Golang and Python.

Impact

Thanks to the implemented recommender system, we made it possible to recommend only the suitable candidates for a particular job. It increased the probability of finding a good match between the available open positions and those qualified available workers in a timely manner. The solution significantly reduced the time spent by recruiters searching through candidate profiles.

Results

Do you want to know more? Contact us.

Get in touch with one of our specialists.

Let's discover how we can help you

Training, developing and delivering machine learning models into production