The Future of Sourcing Is Technical (Scraping and Automation)

booleanstringsBoolean 8 Comments

I think that Talent Sourcing will become more technical. We will have to use scraping and automation to stay productive and competitive. (This is not advice on scraping or automating work on any site in legal terms, of course.)


I anticipate the increased necessity for scraping due to:

1) Growing demand to source for diversity combined with the limitations of “officially” available search filters (notably, on LinkedIn).

Sourcing for Diversity can be challenging since few systems provide “diversity” search filters. We try our best, searching in different ways.

Scraping allows you to be more inclusive because you can search wide(r) and filter results in Excel. You will not need to review each result, only those narrowed by filtering. Results may contain good profiles that you – or others – won’t find with narrow(er) searches or limiting filters.

Any sourcing requiring a “non-existent” filter (not only “diversity”) would benefit from scraping as well. A couple of my recent projects had this challenge:

  • Software Engineers who work for a non-profit (a non-profit is hiring)
  • Business Development Manager in AZ who is Hispanic/Latino (another non-profit, aiming to connect with that community).

2) Professional data being distributed across sites and updated at different times (e.g., Github and LinkedIn)

Scraping can facilitate cross-referencing between sites, combining professional and contact data from each to your benefit.

People aggregators such as AmazingHiring, Entelo, HiringSolved, Hiretual, and others, get professional data in one place and offer some diversity search filters. However, aggregators present two challenges:

  • Data gets outdated fast (it is too expensive to keep updating all of it)
  • Coverage of industries and locations is uneven.

In the end, aggregators can serve as sources but not as a sourcing solution for most projects.


helps in a competitive market (at least) in two areas:

  • Candidate outreach and follow-ups until they respond
  • Recruitment Marketing.

The good news is that you can use “visual” scraping and automation tools that do not require writing code. There is some getting used to UI/UX and understanding each tool’s capacity, but anyone can learn.

How do you find data for scraping? By mastering advanced Google searches and, in particular, X-Ray. You will be finding sites to scrape as well as tweaking X-Ray strings to present scrapable results.

I would be glad to hear what you think – please comment.

Comments 8

    1. Post
    1. Post
  1. Pingback: Sourcing 101: 10 Questions With Top Talent Sourcer Ave Arras

  2. How do you overcome the GDPR compliance in scraping, which states that usage if private emails without permission violate the personal rights?

    1. Post
  3. Pingback: How to Clean Up CSV Export | Boolean Strings

Leave a Reply

Your email address will not be published. Required fields are marked *