For many of us, Deep Web means pages that Google – or any other search engine – can’t find. This is almost true, but not exactly true.
Deep web consists of dynamic links, password-protected sites, pages with no links leading to them, and a few other kinds of “hidden” pages. It turns out (and you may have experienced that) that we see some dynamically-generated paged in out searches on Google. Here is an example that I got while searching for resume, Java, and Swing.
How does this happen? The underlying mechanism is that Google finds “promising” sites with forms and tries to generate pages by “throwing” some keywords to fill those forms. If the resulting pages seem to be “interesting” to Google’s algorithm, Google includes them among the static results.
If you know to to build Google’s custom search engines, you can control Google and only show the deep web results. This is because, unlike standard Google search, custom search accepts special characters such as the question mark.
Bothspecial characters in custom search engine templates and deep web results among other results are cool Google features. I plan to post about Custom Search Engines in more detail soon.
Many of us X-ray LinkedIn from Google. If we are looking for profiles with certain keywords, we use the string
site:linkedin.com inurl:in OR inurl:pub -inurl:dir -inurl:jobs <your keywords here> …
Strangely though, Google forgets to add many words that belong to all LinkedIn profiles as keywords. If you search for “Public profile powered by”, industry, connections, current, past, etc., Google will miss many relevant results. It might be interesting, say, to look for people with few connections using the num-range expression “1..10 connections”; alas, the majority of the results will be missed.
Yahoo does a better job at this; keep in mind though that the Yahoo search engine (initially created by folks at Inktomi) is going away by the end of 2010.
If a LinkedIn member wishes not to have a public profile, LinkedIn promptly removes the info from the Web. However, did you know that what you see marked as private profiles in search results has nothing to do with personal privacy settings? The majority of LinkedIn profiles are public and can be viewed – without upgrading your account. Come to my webinar to get an in-depth look at searching and viewing LinkedIn profiles and contacting prospects on LinkedIn.
* Exploring Private Profiles
* Getting Info on Out-of-Network Profiles
* How to Find More than 100 Members in a Search
* Which Profile Info Can and Cannot Be Found
– LinkedIn Search
– Web Search
* Advanced People Search Dialog Tips
* Selecting Target Lists of Connections
* Email Addresses for LinkedIn Members
* Cross-referencing Using LinkedIn
* Discovering LinkedIn Profiles on Blogs and Twitter
* Custom Search Engines for LinkedIn
As the results of the webinar you will get to the next level in your ability to quickly and easily find the right people on LinkedIn, and make the right choices about upgrading your account.The webinar will be of practical use for recruiters, sourcers, sales, marketing people, and anybody else who would like to find business prospects on LinkedIn.Lecture, demonstration, and Q&A will be included in the presentation. You need to have at least some basic search skills for LinkedIn and for Google to benefit.
Date/time: Tuesday, June 8th, 9 am PDT/noon EDT Length: 90 min Price: $79
As always, my webinars come with one month of Q&A over email.
When you use people search on LinkedIn, sometimes you see the “Out of your network” note (as seen on the left here) and at other times you see that the profile is Private. In both cases LinkedIn suggests that you should upgrade your account to see more. Though Glen Cathey posted an excellent exploration of “private” profiles a while ago, many of us keep thinking that those profiles we find by searching are indeed private, i.e. their owners have changed their settings not to have a public URL (meaning a URL visible to search engines) for their LinkedIn profiles.
The truth is, that private profiles have NO correlation whatsoever with profiles marked “Private” in search results.
I have been exploring out-of-network and “private” profiles lately – and you should, too, before you decide to upgrade your LinkedIn account. I will present a webinar on June 8th, 2010 to show a variety of search techniques for a basic LinkedIn account holder.
Here is an example of a “private” LinkedIn profile found by a friend using LinkedIn people search. Any idea who that might be?
Searching on LinkedIn without any keywords will show some statistics about your current connections. I have looked at mine and here are some numbers that I find interesting – and relevant to the profile of my company, Brain Gain Recruiting. I wonder if others with large networks would have similar distribution – if you take a look, let me know.
United States (788069)
Greater New York City Area (104945)
United Kingdom (99636)
San Francisco Bay Area (58014)
Greater Chicago Area (43605)
Greater Boston Area (40418)
Greater Los Angeles Area (37655)
1st Connections (14102)
2nd Connections (3326977)
Group Members (1536674)
Information Technology and Services (210563)
Marketing and Advertising (124202)
Human Resources (97284)
Computer Software (80838)
Staffing and Recruiting (66489)
Management Consulting (59882)
Financial Services (56159)
Cisco Systems (2450)
Ernst & Young (2283)
If I change the location to the US only, the list is this:
Cisco Systems (1589)
Bank of America (1563)
JPMorgan Chase (1070)
Ernst & Young (7992)
(These lists very much reflect the type of positions Brain Gain Recruiting hires for: consulting, IT, and finances, with our clients being high up on the list.)
University of Mumbai (12426)
University of Phoenix (11876)
Delhi University (9121)
University of Pune (7480)
New York University (6954)
Penn State University (6875)
Bangalore University (6052)
University of California, Berkeley (6045)
University of California, Los Angeles (5970)
Michigan State University (5791)
Osmania University (5704)
School, for the US only:
University of Phoenix (11532)
Penn State University (6675)
New York University (6394)
University of California, Los Angeles (5691)
Michigan State University (5626)
University of California, Berkeley (5545)
Rutgers, The State University of New Jersey-New Brunswick (5509)
Semantic search for recruiters, ideally, would let us enter a job description and get lists of matching candidates.
Here are some sample approaches that either have been implemented in new semantic search tools for recruiters or can be accomplished through search engines like Google (learn more by getting my Google DVD).
Word proximity. If the work managed in near the word people in a resume, we are most likely looking at a manager’s resume.
Abbreviations. It certainly helps to recognize that Sr. is Senior, and PwC is Pricewaterhousecoopers.
Synonyms. If you are looking for a Software Engineer, a Software Developer is a match as well.
Weighted words. You may want to say that some skills are “must-have”, and some are “nice to have”, or, in a more complex system, give higher “weights” to words that are more important.
Keyword clouds. Some keywords may not be in a job description but are common, say, for the industry. Using those terms in a search helps.
Ranking. Search engines rank pages higher if the pages are more popular; we need to see pages that are most relevant, and this is not the same.
If you use people search on LinkedIn and get a person that is “outside of your network” as one of the results, you will see a very minimal set of information and this message:
“Expanded profile views are available only to premium account holders. Upgrade your account.”
But you know what? You do not need to upgrade your account to see much more! Simply click on the public profile URL. This will let you see the profile as if you are logged out of your LinkedIn account. Take a look and you will see lots of info in the public profile – including positions held, companies, and schools, all with dates. You will also see the “show full profile” option – but if you select it, you will get back to not seeing much and an invitation to upgrade your account.
Here is a random example of a LinkedIn profile of a person with zero connections, so none of us would be in the person’s network. Take a look and then click on the “full profile” button. (You might have to log into LinkedIn.) See the difference? The “full profile” shows much less…
I’d say that this is a case where the surface web has more information than the deep web 🙂
Of course, you will not see much if the profile owner hasn’t entered much data – many LinkedIn members don’t. But in this case, being a premium account holder wouldn’t help you either.
Now, if a profile of someone you find is “private”, you will not see a link to the public profile. However, in many cases I was able to find a public link to the profile in question anyway. If you are interested in a private profile that shows up in your search, I am open to helping you find out more information – please email me.
Oh, and I am happy to connect on LinkedIn, here’s my profile.