Three birthday candles for Custom Search

Last Friday was the third anniversary of Custom Search! Here's a photo of our celebratory cake, baked by John Skidgel, our multitalented Senior Interaction Designer. Our cake was baked in between code reviews for our brand new Look and Feel tab. The magnifying glass was a chocolate cake and the handle and cubes are made from marshmallow and rice cereal squares. Luckily, the local cooking store had ready-made fondant in Google colors. :-)

Thanks to you, our users, the Custom Search platform has grown over the last three years: We now have millions of registered Custom Search engines globally, and are serving many tens of millions of search queries per day. Custom Search, now among the top two widgets on the web, powers websites ranging from the New York Times, MacWorld and Orbitz to individual and topical websites. Since the launch of the Blogger gadget a few months back, hundreds of thousands of bloggers use it to help people find information on their sites. Many applications, such as Community Help within Adobe Creative Suite 4 and topical search on About.com, make use of our unique features, such as label refinements, promotions and Linked Custom Search.

In addition to focusing on ranking, on-demand indexing, customizability, ease of use, language support and scalability, we've tried to serve your business needs too — AdSense for Search and Site Search are built on top of the Custom Search platform. These products offer you choice — you can share revenue with Google with topical ads or further control presentation and branding options with the use of XML. We've also developed APIs for our developers and partners.

We wanted to celebrate this week, not just with customized cake, but by treating you to something special. Website owners have always asked about more power and more customization of search results, so we're happy to introduce Custom Search Themes, Structured Custom Search and the Custom Search Wikipedia skin. You can read details about these new features right here on the Custom Search blog. And, starting today, you can follow us at our new Twitter account @googlecse.

Custom Search Themes — Now, with the ultra-configurable Element, you can pick from among a set of convenient layouts and slick styles for your search results. Themes enable well-designed interactive results that you can use instantly on your website and also offer maximum flexibility to further customize every aspect of results presentation. There's also a compact layout for a great mobile experience when using an Android-powered phone, iPhone, iPod Touch or Palm Pre. When you create a Custom Search engine, a mobile home page is automatically created for you.

Structured Custom Search — You know your website and data better than anyone. We already return Rich Snippet metadata attributes in XML results if you provide structured metadata through markup on your pages. Starting today, special attributes, such as Thumbnails and Actions, will be rendered in Custom Search results. More exciting, you can now restrict your search results by specific attributes, based on the metadata that you provide. Custom metadata attributes and custom query restricts will make your searches truly structured.

Wikipedia Custom Search Skin — If you use Wikipedia for research, the Custom Search Wikipedia skin provides highly contextual search results when you search within Wikipedia. You get Google search across all Wikipedia articles as well as topical results based on the links on the Wikipedia article you are currently reading. We think this makes for faster research.

Search is now an expected way to quickly find and navigate information, but there are still many millions of websites out there that don't even have a search box. We hope that these powerful Custom Search tools will help get high-quality search quickly enabled on these websites. After all, you can configure a Custom Search box for your website in minutes!

Posted by: Rajat Mukherjee, Group Product Manager and Nicholas Weininger, Software Engineer

Plug-n-play with Custom Search Themes

Besides great results relevance, one of the most popular features of Custom Search is the ability for you to customize the look and feel of the results to match your website. Our users have always requested more control over results presentation. What's Custom Search without customization?

Today, we're launching Custom Search Themes, which make customized presentation a blast!

You can now select from among a set of slick styles that make the search results look awesome. If you love a style, but need to tweak it, you can customize it further by changing fonts, colors, backgrounds, promotion settings, as well as interactive features such as tabbing and mouseovers. The preview function shows you the effects of your changes instantaneously - WYSIWYG.

You can also select from among a set of convenient layouts for full flexibility on the location of the search box and results - contiguous or in 2 columns - you decide! We've even added a compact layout (with only 4 results) that is useful for websites where space is at a premium and search results will not occupy the whole page. Mobile Custom Search, launched last week, uses the compact layout for rendering results on high-end mobile devices e.g., Android phones, the iPhone, iPod touch and Palm Pre. We automatically create a Google-hosted home page for every Custom Search engine, so you can redirect mobile visitors to this page, since we automatically serve up mobile-optimized results.

There are several results hosting options. The most flexibility is offered by the Element, which renders results inline on a webpage -- your users can quickly review results, narrow results to specific categories with a simple tabbed interface, and dismiss results, all without leaving your webpage. We believe this will make it easier and faster for your website's users to find the information they are looking for. The Element now renders special structured data, e.g., Thumbnails and Actions, that you can provide via structured markup on the page.

With Themes, we're providing great look and feel options for your results out-of-the-box. If you want even more control, you can directly download and manipulate the Cascading Style Sheet (CSS) files that control results styling. You can also refer to the documentation on the AJAX Search API that the Element is built with. Remember - not all customization is pretty, so choose wisely!

We'd love to hear your feedback about Custom Search Themes. If you build out a great customized theme that you'd like to share with us and other Custom Search users, we'd love to see it.

Posted by: Radu Cornea, Software Engineer and John Skidgel, Senior Interaction Designer

Structured Custom Search

Earlier this year, we announced support for Rich Snippets in Custom Search results. If you provide in-page metadata markup, via RDFa, Microformats, or PageMaps, the Custom Search platform extracts the metadata attributes and returns these as PageMaps in your XML results, so you can render these structured attributes in your search results.

Now, our metadata support just got a whole lot better!

Thumbnails and Actions

If you use the Custom Search Element, you can now add publisher-provided links to your search results based on specific metadata markup in your web pages. These special structured data objects, such as Thumbnails and Actions, are generated from the PageMaps you provide. For example, Scribd, one of our partners, provides both Thumbnails and Actions in PageMaps on their pages, so we display both of these in our sample Custom Search engine, as shown below. The Actions in this case are "Download" and "Fullscreen View", but each publisher can provide customized actions, which they can also style via CSS.

In the case of the above document, the PageMap markup for the Thumbnails and Actions is as follows:

<PageMap>
 <DataObject type="thumbnail">
    <Attribute name="src" value="http://i5.scribdassets.com/public/images/uploaded/70327880/D9mDLlYzlw_thumbnail.jpeg"/>
    <Attribute name="width" value="100"/>
    <Attribute name="height" value="130"/>
 </DataObject>

 <DataObject type="action">
    <Attribute name="label" value="Download"/>
    <Attribute name="url" value="http://www.scribd.com/document_downloads/20258723?extension=pdf"/>
    <Attribute name="class" value="download"/>
 </DataObject>

 <DataObject type="action">
    <Attribute name="label" value="Fullscreen View"/>
    <Attribute name="url" value="http://d1.scribdassets.com/ScribdViewer.swf?document_id=20258723&access_key=key-27lwdyi9z21ithon73g3&version=1&viewMode=fullscreen"/>
    <Attribute name="class" value="fullscreen"/>
 </DataObject>
</PageMap>

If you markup your own pages, as shown above, the Element will render your Thumbnails and Actions in your own Custom Search engine.

Structured Search

The above example shows how easy it is for you to add valuable metadata to your pages for use in display in Custom Search results. So, in the sample engine that we used above, here's what the result looks like when you type the query [halloween]. We see a set of nicely formatted results that match the query.

Wouldn't it be nice to use these metadata attributes in your search? We think so too, so we've enabled a way for you to restrict your search results via use of a special operator. Let's say that we wanted to restrict the results to those results that were authored by a specific author, e.g., "lisamorton" (author of the second result above). We can easily do this by adding the following restriction in our search query: [halloween more:pagemap:document-author:lisamorton]. This gives us exactly what we want - documents authored by "lisamorton":

Let's break down the more:pagemap:document-author:lisamorton restriction. The more: operator is what we use for refinement labels. The pagemap: tells us to refine results by specific attributes in the indexed PageMaps. The document-author: qualifier tells us to look for the specific attribute of interest - the document DataObject in the PageMap with child attribute called author. This is followed by lisamorton, which is just the actual value that we are using for the restriction. You can use this open-ended syntax for querying the appropriate DataObjects in your own documents.

The PageMap that represents the document and author information in the markup of the above pages is as follows:

<PageMap>
 <DataObject type="document">
    <Attribute name="title">The Five Scariest Traditional Halloween Stories</Attribute>
    <Attribute name="author">lisamorton</Attribute>
 </DataObject>
</PageMap>

You can provide hooks in your application to filter results by attributes that are important for your users, so users will not have to type these restriction qualifiers directly.

Rich Snippet Preview Tool

Check out the Rich Snippet preview tool. This tool allows you to view not only the Rich Snippets markup recognized for Google web search, but also the additional customized markup that we support in Custom Search. You can immediately see how your web page will be processed after indexing, and what metadata attributes will be returned in PageMaps in your Custom Search results. If there are any errors in your markup, you can fix them right away! Remember - you need to add the &view=cse parameter to this tool to review the additional metadata extracted by Custom Search.

Here's a screenshot from the tool for one of the above results, showing all the PageMap markup recognized by Custom Search:

Many customers have asked for additional structured capabilities, richer snippet presentation and filtering and drill-down capabilities in Custom Search. We'd love to hear feedback from you on these structured data features that offer you more power and flexibility to help users find what they are looking for on your site.

Posted by: Anthony Francis and David Gibson, Software Engineers

A contextual search experience for Wikipedia

Wikipedia users can now configure a Custom Search skin to customize their Wikipedia search experience. Once configured, the skin helps you to search Wikipedia, and for contextually relevant articles, from any Wikipedia page. This can make it easier to find relevant information, especially on Wikipedia pages with many links, and where the topics you are researching are ambiguous. You can find instructions to configure the Custom Search skin at Wikipedia. It works with Wikipedia's Monobook and the Beta Vector skins, and should work on Wikipedia domains globally. Remember that you need a user account and must log in to Wikipedia to use it.

With the skin configured, if you are reading the Wikipedia page on NASA, and do a search for the query [mars], you are presented inline results organized into 3 tabbed groups: All Wikipedia pages, Linked Wikipedia pages, and Linked non-Wikipedia pages. The first tab shows all Wikipedia articles that match, including those about the candy (Mars Bars) and the television series (Veronica Mars). The next 2 tabs provide contextually relevant results that are linked from the NASA page, such as information about various Mars rovers, orbiters, and space labs, as shown in the screenshot.



Here's what's going on under the covers:

Linked Custom Search enables the creation of dynamic search experiences, where the content being searched can be defined on the fly, and can change over time as new information becomes relevant. The Custom Search skin creates a Linked Custom Search engine on demand for every Wikipedia page that you navigate to.

The results from the current Wikipedia domain, as well as the results from the per-page dynamic search engine, are presented inline in tabbed categories via the AJAX search API. You can refine results by the category of choice, and quickly review the results without having to open a new browser window or tab. This happens through the Javascript code in the skin. The skin's CSS defines the look and feel of the results.

As for the page-specific Linked Custom Search engine, it computes the contextual results within the Linked Wikipedia pages (on-domain) and Linked non-Wikipedia pages(off-domain) categories. These two tabs are technically very similar, so we'll just describe how one of them works.

Suppose you're visiting the NASA article and search for [mars]. The Linked Wikipedia tab sends the search query to Google Custom Search, along with a parameter that indicates that the search engine specification is at (view source in browser):

http://googlecustomsearch.appspot.com/wikipedia/spec.do?url=en.wikipedia.org/wiki/NASA

Google picks up this Linked CSE request and uses the above specification and the supplied query. You can simulate this process by visiting:

http://www.google.com/cse?cref=http://googlecustomsearch.appspot.com/wikipedia/spec.do?url=en.wikipedia.org/wiki/NASA&q=mars

A different specification is generated for every Wikipedia page (based on url) by a tiny AppEngine application at http://googlecustomsearch.appspot.com. The specification defines a search engine with two facets, labeled "internal" (Linked Wikipedia pages) and "external" (Linked non-Wikipedia pages). The list of "internal" (and "external") webpages to search over is provided by this line in the specification:

<Include href="http://googlecustomsearch.appspot.com/wikipedia/annotations.do?url=en.wikipedia.org%2Fwiki%2FNASA" type="Annotations"/>

This causes Google to visit the webapp at a new URL (annotations.do). Our webapp now collects links from the NASA article, classifies them as "internal" or "external", and returns the annotations in an XML format. You can see the result at (view source in browser)

http://googlecustomsearch.appspot.com/wikipedia/annotations.do?url=en.wikipedia.org%2Fwiki%2FNASA

Now Google can finish building the Custom Search engine for the NASA article, and compute the results for [mars]. The results are returned to your web browser and displayed in the appropriate tab.

But wait! Our little AppEngine webapp doesn't have the CPU horsepower or bandwidth to scan Wikipedia pages on-demand or in nearly-real-time for thousands of Wikipedia users. Instead, the webapp asks Google to scan the page, via a Custom Search tool called makeannotations. The request looks something like this:

http://google.com/cse/tools/makeannotations?url=en.wikipedia.org%2Fwiki%2FNASA&label=myLabel

After makeannotations returns the list of links in the NASA article in XML, the webapp simply rewrites the XML according to the domain of each link.

Since we are creating the per-page search engines on demand, there can sometimes be a short delay in the creation of the search engine, e.g., for new or obscure pages. However, for popular Wikipedia pages, these definitions should be cached, and you should see no delays. In fact, we use a ping method to load up the Custom Search engine in advance before you search. Remember that if there are not many links on the Wikipedia page you are searching from, you may sometimes find no matches for linked pages.

We've open sourced the code for this application. Feel free to work with it. Feel free to extend the skin beyond Monobook and Vector. We built this skin with the help of Wikipedia, and hope that you will provide feedback on your experience. You can also provide your feedback directly to Wikipedia.

Posted by: Paul Komarek, Software Engineer and Jeffrey Scudder, Developer Programs Engineer

Google Custom Search for your smartphone

In our recent Product Ideas survey, one of the most-requested features was about enabling a mobile version of Custom Search. Guess what: It's here! As announced on the Google Mobile blog, Custom Search now enables a rich interactive mobile experience on high-end devices such as Android-powered phones, iPhone, iPod touch, and Palm Pre.

Try it out on your phone right now: search for user-generated content (e.g., search for [maradona]) from sites like Wikipedia, Knol, etc., or learn all about the latest and greatest Custom Search features on our product's mobile search engine (e.g., search for [snippets]).

Custom Search engine results can be embedded in your own site, but website owners can also choose to have Google host the search results page. You can now redirect visitors using these high-end mobile devices to Google-hosted search results, and we will serve up search results optimized for these devices. Better yet, we create a Google-hosted home page for every Custom Search engine that we create (even if you decide to host the results on your own website). Starting today, users who arrive at the home page for your Custom Search engine will be automatically served pocket-sized Custom Search results, along with your search engine customizations, like your labels and promotions.

All you need to do at your own end is redirect users who arrive at your website on one of the supported mobile devices to the hosted Custom Search home page. Another alternative is to provide a button or link on your web site that directs people to your Custom Search home page, whose URL looks like (you will need to replace the ID after "cx=" with your own Custom Search engine ID):

http://www.google.com/cse/home?cx=013315504628135767172:d6shbtxu-uo

Of course, if you'd like to serve these mobile results from your own website, you can replicate the functionality of the mobile home page on your own website. This page uses the Custom Search Element. If you do, you will still need to take care of the redirection at your website so that mobile users receive mobile-friendly search results automatically.

If Google web search is the entry point for navigating the web, we hope you'll use this interactive Custom Mobile Search entry point to help people navigate your own web content when they need information on the go!

Posted by: Radu Cornea, Software Engineer and John Skidgel, Senior Interaction Designer

Google Sites turns Custom Search on

Google Sites offers users a very simple way to create and publish web sites. Multiple users can collaborate to manage website content as well as add various gadgets into their websites. Google Sites now allows the addition of Google Custom Search as a search option.

In the following example, we add a Yoga Search Custom Search experience to the Yoga for Life web site in a few simple steps. (The example assumes that the Yoga for Life website is hosted by Google Sites, and that the Yoga Search has already been created using Google Custom Search.)

1. Login to your Google Site, and select the "Manage Site" option in the "More Actions" dropdown.
   
   


2. Select the "Site Layout" option from the Navigation menu on the left. Click the "Configure Search" button.
   
   


3. Select the Advanced tab in the "Configure site search" dialog. To add a new search provider, click on the "Add provider" button.
   
   


4. Now, we'll need to get information about the Custom Search engine we want to use. Open up another tab or browser window and login to the Custom Search control panel. Find the URL for the search engine you want to use by clicking on it's name in the list called My search engines. Copy this URL; you will use it in the next step. In this example, we'll use the URL for Yoga Search.
   
   


5. Go back to the Google Sites search provider dialog. Select "Google Custom Search" as the provider type. Type in the Name, e.g., "Yoga Search" for the search option you'd like your users see on your site, and paste the Custom Search Engine URL that you obtained in the earlier step. Leave the Show results in site option checked. Click OK.
   
   


6. You can change your default provider by clicking on the "General" tab in the "Configure site search" dialog and selecting "Yoga Search". Make sure to Save your settings. You're done.
   
   


7. Your users will see Yoga Search as the default search option in the search box on the Yoga for Life website. Search, and your results from the Yoga Search Custom Search engine will show up inline, with the look and feel of your site.
   
   

Try it out for your site, with your own Custom Search engine! As always, we'd love to hear your feedback on new features in the user group.

Posted by: Scott Johnston, Senior Product Manager, Google Sites and Rajat Mukherjee, Group Product Manager, Search