Custom Search Engines Hack: Get 1,000 Results

booleanstrings Boolean 7 Comments

google_custom_search_engine

Google’s Custom Search Engines (CSEs) can be useful in many ways. They provide a way to hide advanced search operators from your colleagues who are less technically inclined; they don’t bug advanced researchers with annoying Captchas; and they provide some interesting possibilities beyond those of “regular” Google.

Unfortunately, currently the “official” limit of the number of the search results in a CSE is 100 , with the maximum of 20 results per page. Given that CSE’s have their own ways to pick the results from Google’s index (if you have played creating and testing CSE’s you’d know what I am talking about), these numbers seem too limiting for any serious research.

Hooray! I have discovered a “hack”, that allows to get up to 1,000 search results, up to 100 per page in CSEs, that I am about to share. The way to do this is to use an old web interface that has been abandoned (and is not documented any longer, which makes it slightly challenging figuring out how to customize its look and feel). I do hope it stays on!

Let me get straight to the point. A standard public link to a CSE looks like this:

https://www.google.com/cse/publicurl?cx=009462381166450434430:awjijlwzhjs

The highlighted piece takes us to a CSE, hosted by Google; the long string after cx= is the unique CSE ID. The above is a public link to a CSE that I created, Document Finder (Storage) that looks for documents stored in a number of sites such as scibd, slideshare, etc.

Try this search, for example

employee directory

– and get 100 results, which is the maximum.

Now, if you use a different URL template, the CSE takes you to a different User Interface – and it’s there that you get many more results – up to 1,000!

The alternative URL looks like this:

http://www.google.com/custom?cx=009462381166450434430%3Aawjijlwzhjs&num=100…

The trick I discovered is to use this different Google-based URL, with the word “custom”. Now, try this:

employee directory

Currently, this search provides 1,000 results. That is quite unusual even for the regular Google these days! Here is what it looks like (there are some very interesting results this search provides by the way):

cse-1000

If you’d like to get existing CSEs to work this way: use the above format. Additionally, add &num=100 to set the number of results per page to 100; &filter=0 to see “all” results. You can even search by a date range or verbatim.

Check out my collection of Custom Search Engines here on the blog. I will adjust the links shortly there as well.

 

 

 

 

Comments 7

  1. Awesome discovery – I have always restricted my self thinking that Google will allow only 100 results in CSE. Not even ever thought about how to hack it.

  2. It is very helpful, but I would like to add a notification:

    Not only ‘custom’ (‘/cse/public’) is the difference between the two URLs, but the http(s) protocol, too. I think, it should be also highlighted in the article.

    1. Post
      Author
    1. Post
      Author

      Hi Robin, you are right, it doesn’t. So don’t many of the past posts, because things change fast in our profession. I do try to update some posts and definitely pages, but it would be a huge effort to go back 10+ years and update the details. However, you can count on always up-to-date sourcing classes at https://sourcingcertification.com/library/.

Leave a Reply to Robin Cancel reply

Your email address will not be published. Required fields are marked *