Home delivery of Custom Search to webmasters

Posted by: Radu Cornea, Software Engineer and Rajat Mukherjee, Group Product Manager

Until today, when you wanted to add Google Custom Search to your site, you had to head directly to our Custom Search Engine site. Although creating your own Google-powered search engine only takes a few minutes, we thought it might be easier if we delivered this functionality right to where many of you manage your sites online. Choice is a beautiful thing.

If you own a website hosted on IPOWER, StartLogic, PowWeb, FatCow, BizLand, EasyCGI and several other web hosting providers, you can now configure Custom Search right in your hoster's management console - these hosters now support both Webmaster Tools and Custom Search. It's like having pizza delivered to your door instead of having to drive over to the pizza parlour!

When you sign up for the service, you get the following benefits, all from the comfort of your hoster's control panel:

• a Google account is automatically created for you - you don't have to remember additional passwords or login to use these Google services
• you can generate search engines for your websites by simply identifying your domains
  
• shortcuts are provided to manage your Webmaster Tools account and your Custom Search Engines
• Sitemaps for URLs on your sites can be automatically submitted to Webmaster Tools for better indexing on Google.com
• your Custom Search engines will automatically recognize and use the submitted Sitemaps for enhanced indexing coverage
• you can make money by associating your Adsense publisher ID with your search engines and sharing advertising revenue with Google
  
• you can upgrade your search engines to use Site Search for further control over branding, presentation (via XML results) and On-Demand indexing

If your site is hosted by one of the above providers, give it a try - go to your hoster's console and add a search box to your site if you don't already have one. High quality search improves user experience, and also directly increases user engagement and improves conversions on web sites.

Additionally, let us know if there are things about this integrated service offering that we can improve. Our programmer's guide provides more details about the new Custom Search APIs that allow programmatic creation and management of Custom Search engines. If you are a web site hoster interested in incorporating these Google webmaster services into your platform, please contact us.

Searching the Google Timeline

Posted by: Tyler Morse, Software Engineer and Dylan Casey, Product Manager

When we set out to build a timeline to commemorate Google's 10th birthday, we wanted to add an experimental search feature. We may be a little biased, but because we needed to search a specific subset of the web, and needed customized presentation, we decided to use Google Site Search, which is powered by the Custom Search platform. Using Site Search, the team designed a customized user interface and user experience, tightly embedded within the timeline application, in a few days.

The Google Timeline data is stored and served up as static files of JavaScript objects and arrays. While this was very efficient for the Timeline JavaScript application, we needed to massage the data in order to get it indexed and searchable, as well as tie it back to the timeline objects.

We generated static HTML files for all timeline events. The filenames have identifying data in them that allow us to correlate a search result with the original event; this way, we can translate a user click on a search result to the appropriate timeline event. Since we wanted explicit control of event dates, we also encoded the event date in each HTML file.

We use Site Search's flexible XML interface for retrieving the results. When we get back search results, we parse the XML data, extract specifically the information we need, and render these results.

Since we did not want to take users away from the timeline application, we presented the results inline. As we had limited real estate available, we only displayed 3 results at a time. As shown in the screenshot above, we added pagination to the results section, and the ability to sort the events by event date.

We couldn't treat our search results like typical URLs, as on Google.com. We wanted to show users the event within the actual timeline when they clicked on a search result, instead of navigating them to the indexed static HTML file. Clicking on a result provides an interactive experience, and brings up the specific timeline event.

The end result: a Google search for Google's history, custom fit to meet the specific needs of our timeline, just before we blew out the candles on our cake.

On-Demand Indexing for a fast-moving web

Posted by: Tom Duerig and Nicholas Weininger, Software Engineers

Earlier this year when we launched Google Site Search, and AdSense for Search started using the Custom Search platform, we created a special Custom Search Engine (CSE) index for enhanced indexing. Webmasters could submit Sitemaps in Webmaster Tools, and the Custom Search platform indexed URLs from these Sitemaps into a special CSE index for more comprehensive coverage.

This solved an important problem: the need for enhanced index coverage for site search. The special CSE index, in addition to the Google index, enabled us to search deeper on selected sites, providing higher search quality.

Today, with our launch of On-Demand Indexing, we're addressing the need to provide fast indexing for your important, frequently-changing content. On-Demand indexing allows anyone with a Google Custom Search Engine or Google Site Search to identify and tell us about new pages or recent site changes by submitting a Sitemap to our Webmaster Tools. You can then select this specific Sitemap in the Indexing tab of your CSE control panel and hit the new "Index Now" button. We will immediately schedule the relevant pages for crawling and indexing, and these pages will be included in your search results usually within 24 hours - often much faster.





We realize that new or frequently updated content is often crucial to a website. In order to give you more control over how you use the On-Demand Index resources allocated to you, we will honor the priority and lastmod attributes that you provide in your Sitemap - this means you can identify what URLs should be given preference. Please ensure that your Sitemap metadata is up-to-date.

On-Demand Indexing uses a special index that is designed just for Custom Search. The main Google search indexing process is separate and uses different selection criteria, so submitting pages for On-Demand Indexing will not make them appear any faster in the main Google index, or impact ranking on Google.com. For more information on On-Demand Indexing, as well as the allocation per CSE, please refer to our FAQ.

We are always listening at our Custom Search discussion group, so please continue to tell us what features you'd like to see added to Custom Search.

Knol - now with Custom Search

Posted by: Cedric Dupont, Product Manager

Knol is a project aimed at helping people share their knowledge. Knols (units of knowledge) are authoritative articles written by people about a specific subject, ranging from tooth pain to solar energy to buttermilk pancakes. With all of these knols to browse, our readers have been begging us for a more powerful search tool. Well, today we have good news - we've added the power of Custom Search to Knol.

Custom Search gives us all of the speed and relevance of Google's search technology, but required none of the hard labor that went into making Google search what it is today. With Custom Search, Knol visitors will have a fast search experience that features all of the bells and whistles Google searchers have come to love (including our advanced spell checker). When you search within Knol, the search results look a little different from Web Search results (see this search for knee surgery, for instance). We were able to maintain that distinct feel for Knol search by taking advantage of the Custom Search XML results format (via Google Site Search) and creating the search result display that we wanted.

We want to hear from you if you have feedback on Knol search. Please leave your comment on our help page.

Synonyms for your Custom Search Engine

Posted by: Victor Wang and Bartlomiej Niechwiej, Software Engineers

With our launch of Google Site Search, we added a new feature to the Custom Search platform: custom synonyms. Here's how this feature can be used to improve the quality of your Custom Search Engine (CSE).

How can custom synonyms help?
Synonyms help by finding documents with relevant related terms and ranking them higher. Synonyms can help to alleviate the mismatch that often occurs between the queries that users type, and the actual words, phrases and concepts used in the documents being searched. Custom Search, of course, automatically takes advantage of synonyms used in Google Web search. In addition, Custom Search goes one step further: we allow you to explicitly define custom synonyms that are specific to your web site, community or topic of interest.

To illustrate situations where CSE custom synonyms can help, we created 2 CSEs that both search content from the Palo Alto Medical Foundation (PAMF). The first CSE does not have custom synonyms enabled, while the second CSE has a few custom synonyms enabled:

Terminology: The queries people use sometimes don't match up with the words and phrases used in the content being searched. During allergy season, for example, many people look for information on "hayfever", but the results without synonyms aren't that great since the web pages we're searching across don't necessarily use this specific term. However, if the technical phrase "allergic rhinitis" is added as a synonym for "hayfever", the results with the synonym are far better.

Acronyms: Acronyms often stand for different terms in different contexts. This is especially true within organizations, where acronyms are used frequently. In such cases, it may be possible to improve retrieval via the use of synonyms. Thus, "PAMF" refers to "Palo Alto Medical Foundation", and adding this synonym improves search results: searching for "PAMF" without synonyms gives only a few relevant results, while the CSE with the synonym returns more relevant results.

Community: Within specific target user communities, words or phrases have different usage and significance. When a patient is looking for "hearing doctor", adding the synonym "audiologist" to the CSE provides much better results while the results in the CSE without synonyms are not optimal. Conversely, if a doctor is searching for "somnambulism", she finds no results at all in the CSE without synonyms, but much better results via addition of the synonym "sleepwalking" to the CSE. Synonyms can therefore be used to improve the experience of specific classes of users of a web site.

How can I add synonyms to my Custom Search Engine?
The CSE administrator can control the set of synonyms used by uploading a synonym dictionary that is specific to the domain and website. The synonym dictionary can include alternate words or phrases for common search queries. The following steps show how to add the synonyms for the Palo Alto Medical Foundation CSE.

1. Download the existing CSE context file through "Control panel"->Advanced->"Download context"


2. Add custom synonyms to your search engine. The synonym dictionary is uploaded as part of the context XML file. Here are the synonyms we added for the above examples:

   
   <customsearchengine>
    <title>...</title>
    <description>...</description>
    <context>
     <backgroundlabels>...</backgroundlabels>
   
     <synonyms>
      <synonymentry word="hearing doctor">
       <synonym>audiologist</synonym>
      </synonymentry>
      <synonymentry word="hayfever">
       <synonym>allergic rhinitis</synonym>
      </synonymentry>
      <synonymentry word="somnambulism">
       <synonym>sleepwalking</synonym>
      </synonymentry>
      <synonymentry word="pamf">
       <synonym>Palo Alto Medical Foundation</synonym>
      </synonymentry>
     </synonyms>
   
    </context>
   </customsearchengine>



3. Upload the context file through "Control panel"->Advanced->"Upload context"

A few notes:

• CSE synonyms are unidirectional, not bidirectional. Thus, a context file with

  <synonymentry word="migraine"><synonym>headache</synonym></synonymentry>

  defines "headache" to be a synonym for "migraine". However, if you also want "migraine" to be considered as a synonym for "headache", you need to add a separate SynonymEntry to the XML, as follows:

  
  <synonymentry word="migraine"><synonym>headache</synonym></synonymentry>
  <synonymentry word="headache"><synonym>migraine</synonym></synonymentry>
  



• In the current version, the synonym dictionary can only be uploaded/downloaded as a part of the context XML file. We hope to make this much easier in the future.


• We allow up to 500 individual synonyms for a given CSE.


• Each word can have no more than 10 synonyms. If there multiple synonyms, the query will be expanded to include all synonyms uniformly.

If you are using custom synonyms in your CSE, we hope to get feedback from you about what improvements we can make.

Connecting with the Adobe Community with Google Site Search

Posted by: Tom Duerig, Software Engineer

We love Google Site Search. And when working with our customers, we often discover new and interesting ways to apply our technologies to their business needs.

With Adobe's latest launch of Creative Suite 4, (which includes some of their most popular desktop applications, such as Adobe InDesign, Illustrator, Photoshop, Dreamweaver and Flash), we had the chance to work with their team to integrate Google Site Search into their product in a way that truly blurs the line between off-line and online content.

Adobe is using Google Site Search as part of Creative Suite 4's Community Help to connect suite customers to Help content on and off Adobe.com, including tutorials, technical support, online product help, videos, articles, tips and techniques, blogs, examples and other resources.

In chatting with Adobe Product Manager, Mark Nichoson, we learned that Adobe chose Google Site Search for a variety of reasons. First, we learned that user experience and consistency are essential for Adobe customers. After several studies, the Adobe Creative Suite Team concluded that their customers would use Google Site Search to access information on their products.

At the same time, the nature of Adobe Community Help is to align, share and offer the best available information online. Google Site Search allowed this to happen by aggregating more than 4000 URLs coming from the active members of their customer base (bloggers, partners, developers, webmasters and others).

This is just one of the ways Google Site Search is connecting customers and their users with the information they need -- whether they're within their applications or on their websites. We can't wait to see what unique Site Search implementations come up next!

Beefed-up developer guide

Posted by: Kevin Gargoyle Lim, Technical Writer

We've updated the developer guide to include the more advanced Custom Search features, such as synonym expansion and integration with Subscribed Links. These features let you trick out your search engine and give your users a richer search experience. Synonym expansion lets you expand a user's search term (such as "running") to include its variants (such as "jogging" and "sprinting"), so users will not need to search for each variant. Subscribed Links enables you to directly answer your users' questions by promoting a specific result at the top of the results page. You can create your own result text and define sets of queries that would trigger subscribed link results.

The developer guide may not be edge-of-your-seat material, but it does now come with a freebie glossary and a search gadget that you can add to your iGoogle page. We're still tinkering with the documentation and bulking it up with more juicy tips, so keep coming back to check our progress.

Happy reading!