Custom Search Engine APIs

Posted by: Matt Wytock, Software Engineer

A couple of weeks ago we blogged about a new feature and a new kind of Custom Search Engine (CSE) that you could create on the fly. Today, we thought we'd dig a bit deeper and describe the underlying infrastructure that powers this. With our new Linked CSEs, we are exposing the API to create and control CSEs.

Until now, you created a CSE either by using the wizard or by writing an XML file and uploading it to Google (via the "Advanced" tab on the control panel). To change any aspect of the CSE, you had to either use the control panel or upload the new XML specification. This imposed several limitations:

• Creating and maintaining a CSE was a manual process.
• It was difficult to create a large number of CSEs.
• It was difficult to use other data sources such as iCal, RSS, Google Base, etc. to programmatically create CSEs.

The search box code for these CSEs (found on the "Code" tab in the control panel) includes a "cx" parameter with every search request (for example, <input type="hidden" name="cx" value="005946352831473999820:qs1idu8ptku" />), which specifies an internal identifier for the CSE.

Linked CSEs overcome these limitations. In short, you can now specify your CSE using a "cref" parameter that points to a URL, anywhere on the web. You update this URL at your end and don't have to upload it or edit your CSE using our tools. The URL can take arguments to produce dynamic CSEs, based on the current page, the current user visiting your site, etc. You can see this in action on our "on the fly" demo page: when you type "http://www.cs.berkeley.edu/~russell/ai.html" in the form text field, the javascript on that page constructs a "cref" parameter that contains http://www.google.com/cse/tools/makecse?url=http://www.cs.berkeley.edu/~russell/ai.html. This URL (visit it!) contains an XML specification for a CSE. You can use any script you want, or reference a static file, when creating your CSE. And there's nothing special about our makecse example script: we're hoping that our developers and the developer community will build many other such CSE-generating tools.

How does this work? With Linked CSEs, you designate a CSE specification URL with each search request (as a hidden form field in your search box HTML code). Google retrieves the CSE specification from the URL when your user searches in the CSE. We cache and refresh the results so that only the first search to your CSE incurs any delay. The flexibility to specify how your search engine should behave, just when your user is doing the query, using whatever data sources you want, opens up many possibilities:

• You can use our makecse tool to generate CSEs from different sources of links:
  • HTML: http://www.cs.berkeley/~russell/ai.html
    
  • RSS: http://reddit.com/.rss
    
  • ATOM: http://lipstadt.blogspot.com/feeds/posts/default
    
  • OPML: http://medicalconnectivity.com/gems/blogroll.opml
    
• You can combine multiple sources of links using our makeannotations tool and the <Include> tag. For example, its easy to create a search engine from the links on the front pages of techmeme, slashdot and digg.
  
• You can write your own tools to produce <Annotations /> XML from other data sources such as Google Calendar or iCal feeds, Google Base or any other structured source of information.
  
• You can automatically generate any number of CSEs, each possibly tuned to a particular user. For example, we've created a sample that builds a CSE from a user's digg.com friend network and submissions using the Digg API. Try it out and view the source. This makes use of two simple python CGI scripts:
  • diggannos.py generates <Annotations> from the specified user's submitted stories
    
  • diggcse.py generates <GoogleCustomizations> from the specified user's friend network. For each friend, it generates an <Include> element pointing to the appropriate diggannos.py URL

You can test any Custom Search Engine XML by going to http://www.google.com/coop/cse/cref and entering the URL. Putting a search box on your site is as easy as copying a small bit of HTML code and modifying the "cref" parameter.

Linked CSEs are a very big step for Google Custom Search. We hope you will find them as cool as we do. As always, thank you for your support and keep the feedback coming.

Finding your community

Posted by: Vrishali Wagle, Software Engineer

Since we launched Custom Search Engines (CSEs) last year, we have seen search engines created on a variety of topics. Many of these search engines are open to volunteers and would love to have you contribute to them.

Now, it's easier for you to find all of these CSEs. Go to the Featured Examples page and type relevant words into the search box at the top of the page. We will search for these words in the fields for the search engine's name, description, keywords, and popular queries. If you're interested in finding a search engine to contribute to, search specifically for search engines that allow volunteers. For instance, if you're most interested in non-profit organizations, search only for non-profit search engines. You get the gist.

All of this searching over search engines is done via Google Base. It's the place to submit your content and make it searchable and accessible on Google. You can look at the hundreds of custom search engines that are related to cricket, podcasts and lyrics. We also have search engines on pet rats and many flavors of legal search.

Please note that only a small fraction of existing CSEs are included in this search. We've filtered search engines based on quality, recency, traffic and other metrics.

We hope this new Custom Search Engine search feature enables you and other CSE creators to find search engines more easily, and that it ultimately provides you with better access to information in your community.

Status Message and Older Post picture link

This tutorial covers 2 aspects. First, we shall customize the template to remove the status message “Showing posts for query link. Show all posts” appearing at the top of the posts when you enter a search word in the Blogger Navigation bar. Secondly, in the previous article, we discussed the ways to remove or hide the Newer Posts, Home, and Newer Posts links at the bottom of every post. We

Post Footer – Customize Template (I)

In this tutorial, we discuss the various ways you can customize the Post Footer and change the Blogger template. The steps will cover how to change the “Posted by author”, “Comments”, “Labels” links in the Post Footer, replacing them with other names or pictures.This is how a typical Post Footer looks like. It includes the author's name, time of the post, number of comments, number of backlinks

Post Footer – Customize Template (II)

We continue from where we left off in Post Footer – Customize Template (I). In the first part, we went through how you can change and customize the appearance of the “Posted by author” link and replace it with another name or picture. You could also remove the timestamp if you want. In this part, we continue with customization of the “Comments”, “Labels” and other icons shown in the Blogger

Custom Search on the fly

Posted by: Rajat Mukherjee, Product Manager

Starting today, there's a new feature that makes Custom Search Engines (CSEs) even easier to create and keep up to date.

You can now create a CSE by simply placing a small piece of tailored code on a page on your site. With that one piece of code, Google's search technology will automatically include in your new CSE all of the sites you have linked to from that page, creating a dynamic, powerful and tailored search experience really quickly. Moreover, your new CSE will update itself periodically to include any new links added to that page.

So, if you have a blog or a directory-like site and don't feel like listing all of the URLs you want to search across, you can leave the work to us. With this new feature we'll automatically generate and update your CSE for you. For example, try the query 'sculpture' on this CSE dynamically created from a page of links to kids museums or the query 'planning' on the search engine about Artificial Intelligence we created from the page of links at Berkeley.

Pretty cool, eh? We think so too. There are many powerful things you can do with this new feature, and in the near future we'll be talking about different possibilities. In the meantime, however, feel free to get your dynamic Custom Search Engine up and running. We'll be back in an instant.

Keep the feedback and great ideas coming!

Submit RSS Feeds and Pings

In this article, we list all the Feed Directories to which you can submit your feeds for Free. We shall also list the Ping services that you can use to ping whenever you update your Blog. This should help increase awareness of your new articles and attract more readers to your Blog. As the list is ever-growing, you might want to consider bookmarking this page as we shall update the list whenever

Scrolling Text - Marquee HTML Code

This HTML term Marquee is used to create a scrolling or sliding text. The text can move across the screen horizontally or vertically at a certain speed determined by you. Note, however, that while it looks nice to have scrolling announcements, advertisements, quotations, or links, having too much animation in your Blog may distract and irritate your readers. We had earlier created a line of