Scrape X-Ray for Research Including Diversity

booleanstringsBoolean 1 Comment


I have a demanding client who keeps sending me challenging sourcing requests, needing more than Boolean Search in LinkedIn Recruiter can achieve. (I love those.) Her last three requests have been – all in Europe – to find:

  1. Frontend Developers who have LinkedIn recommendations.
  2. Backend Developers who used to work as Research Assistants and have good grades (like “cum laude”).
  3. Software Engineers with under ten years of experience who are Diversity (not just women, she insisted).

The expectation was to get lists with hundreds of profiles. I was able to deliver, but it involved a lot of head-scratching.

You cannot search for Recommendations with any LinkedIn account. You cannot look for school grades. If you search for an OR or “good grades” terms, you will only find a small portion of those who have put it elsewhere on the profiles (good for them!). And, LinkedIn does nothing to help search for Diversity.

Since many diverse categories represent the minority of professionals in the desired professions, just searching on LinkedIn and screening the results for Diversity is too time-consuming. I had to go with X-Ray.

So, for Task 1, I X-Ray LinkedIn, patiently, by country, for the words “recommendations received” to be there. I scrape results with Instant Data Scraper and filter out false positives. Now I have a list of promising LinkedIn URLs. A way to go is to paste the list to SalesQL‘s (brilliant!) upload function. Its export features more fields from profiles than I have seen anywhere else. (An alternative is Phantombuster.) As a result, I have a rich Excel file, which I clean up, sort, filter, and select my prospects, whom I can email. Because of this additional step, my search strings can be imprecise, and I get results which won’t surface in a tighter search.

Task 2, same story with various “cum laude” words.

Task 3 is more complicated because I need to find “Diversity Indicators” for each type of Diversity. But the principle is the same. I start with researching Diversity Identifiers such as women’s names, relevant organizations, diversity schools, etc. Note that simple scraping helps here as well. I can ask Google questions and get lists of terms as Featured Snippets. Then I use an OR of the terms both on Google and LinkedIn.

Another type of sourcing to be done with scraping is searching for job stability, sadly, never offered by any sites.

It is time to update your scraping skills! Join me for the upcoming class Web Scraping For Recruiters on January 5th, 2022.

Comments 1

Leave a Reply

Your email address will not be published. Required fields are marked *