experfy logo
Candidate recommendation engine
Building a recommendation engine capable of using Natural Language Processing to automatically parse thousands of candidate resumes and recommend them to a job opportunity
About the client
Experfy is a startup that kicked off from the Harvard Innovation Lab in Boston and Deloitte’s hiring partner for data science and artificial intelligence projects. They provide top of the notch experts for demanding projects. Experfy has a pool of specialists who can be recommended for new positions at Deloitte’s clients.
The goal
Initially the client’s talent team could manually recommend the best candidates for any given project but this soon became a problem as the expert’s and project’s list began to grow. We were asked to build a recommendation engine that could help find the best professionals for any given project using their skills, past experiences and education.
goal tree
The Data
Although the client had rich information, it was not structured in a way that we could easily use. For example, we had to read uploaded resumes to extract some relevant skills or past experiences. We not only had to build a recommendation engine but also a natural language processing system to understand both recruiters and experts. Business rules also had to be applied, filtering candidates by nearby locations, rates and availability.
Our Solution
Extracting and matching skills from a project and a resume is not an easy task. It might be easy for a human to understand that Mysql and Postgres are really similar skills but it’s not easy to teach that to a machine. The problem escalates when you have an ever expanding database of candidates with new skills. At this point, a rules engine is just not up to the task. another challenge was we had to make the recommendations in real time.
Our work involved combining skills embeddings trained for this specific purpose with a ranking system for which we ended up using Yandex’s Catboot. The system’s features are actually more complex than just a recommendation (we use machine learning for multiple tasks that help reduce manual work). Different requirements forced us to use more precise metrics such as word mover’s distance or wmd when we needed to be accurate, and simpler metrics when we needed to be faster. These additional features added to the system allowed us to include Google’s BERT question answering.
solution diagram
We teamed up with the client’s IT team to deliver a microservice that was integrated with the existing architecture, allowing the client to scale up operations:
solution diagram
Technologies used involved Python, Golang and Tensorflow. All got deployed to GPU servers as speed was important. We implemented a CI&CD pipeline using Docker and Kubernetes in AWS. Automatic tests were implemented in both Golang and Python.
technologies logos
Thanks to the implemented recommender system, we made it possible to recommend only the suitable candidates for a particular job. It increased the probability of finding a good match between the available open positions and those qualified available workers in a timely manner. The solution significantly reduced the time spent by recruiters searching through candidate profiles.
Do you want to know more? Contact us.
Or drop us an email at
[email protected]
Got a project?
Let’s talk
Tell us about it