For many of us, Deep Web means pages that Google – or any other search engine – can’t find. This is almost true, but not exactly true.
Deep web consists of dynamic links, password-protected sites, pages with no links leading to them, and a few other kinds of “hidden” pages. It turns out (and you may have experienced that) that we see some dynamically-generated paged in out searches on Google. Here is an example that I got while searching for resume, Java, and Swing.
How does this happen? The underlying mechanism is that Google finds “promising” sites with forms and tries to generate pages by “throwing” some keywords to fill those forms. If the resulting pages seem to be “interesting” to Google’s algorithm, Google includes them among the static results.
If you know to to build Google’s custom search engines, you can control Google and only show the deep web results. This is because, unlike standard Google search, custom search accepts special characters such as the question mark.
Both special characters in custom search engine templates and deep web results among other results are cool Google features. I plan to post about Custom Search Engines in more detail soon.