Custom Search Engines Hack: Get 1,000 Results

booleanstrings Boolean

google_custom_search_engine

Google’s Custom Search Engines (CSEs) can be useful in many ways. They provide a way to hide advanced search operators from your colleagues who are less technically inclined; they don’t bug advanced researchers with annoying Captchas; and they provide some interesting possibilities beyond those of “regular” Google.

Unfortunately, currently the “official” limit of the number of the search results in a CSE is 100 , with the maximum of 20 results per page. Given that CSE’s have their own ways to pick the results from Google’s index (if you have played creating and testing CSE’s you’d know what I am talking about), these numbers seem too limiting for any serious research.

Hooray! I have discovered a “hack”, that allows to get up to 1,000 search results, up to 100 per page in CSEs, that I am about to share. The way to do this is to use an old web interface that has been abandoned (and is not documented any longer, which makes it slightly challenging figuring out how to customize its look and feel). I do hope it stays on!

Let me get straight to the point. A standard public link to a CSE looks like this:

https://www.google.com/cse/publicurl?cx=009462381166450434430:awjijlwzhjs

The highlighted piece takes us to a CSE, hosted by Google; the long string after cx= is the unique CSE ID. The above is a public link to a CSE that I created, Document Finder (Storage) that looks for documents stored in a number of sites such as scibd, slideshare, etc.

Try this search, for example

employee directory

– and get 100 results, which is the maximum.

Now, if you use a different URL template, the CSE takes you to a different User Interface – and it’s there that you get many more results – up to 1,000!

The alternative URL looks like this:

http://www.google.com/custom?cx=009462381166450434430%3Aawjijlwzhjs&num=100…

The trick I discovered is to use this different Google-based URL, with the word “custom”. Now, try this:

employee directory

Currently, this search provides 1,000 results. That is quite unusual even for the regular Google these days! Here is what it looks like (there are some very interesting results this search provides by the way):

cse-1000

If you’d like to get existing CSEs to work this way: use the above format. Additionally, add &num=100 to set the number of results per page to 100; &filter=0 to see “all” results. You can even search by a date range or verbatim.

Check out my collection of Custom Search Engines here on the blog. I will adjust the links shortly there as well.