Delivering the goods

[DVTimes]

http://www.guardian.co.uk/online/sto...117818,00.html


Delivering the goods

There's no doubting Google's power and popularity. Yet few of us use the search engine effectively. Jack Schofield offers some tips

Thursday January 8, 2004
The Guardian

Google is now the world's most powerful website, and if it goes public this year, its young founders, Larry Page and Sergey Brin, will become extremely rich. Their five-year-old company has already cracked its biggest problem, which is how to make pots of money from selling advertising space without carrying any banner ads. And while there are other places to search the web, most websites are now dependent on Google for a large proportion of their new visitors. The question that drives all but a few commercial webmasters today is: "How do I change my site to make it appear on the first page when someone searches Google?"
What is even more impressive is that Google has achieved its supremacy by word of mouth: by delivering what users want. That has helped it retain users' confidence while doing things that might have raised concerns about invasion of privacy elsewhere. For example, Google almost certainly knows more about you than you would tell your mother. Did you ever search for information about Aids, cancer, mental illnesses or bomb-making equipment? Google knows, because it has put a unique reference number in a permanent cookie on your hard drive (which doesn't expire until 2038). It also knows your internet (IP) address.

Google's privacy policy says that it "notes and saves information such as time of day, browser type, browser language, and IP address with each query. That information is used to verify our records and to provide more relevant services to users. For example, Google may use your IP address or browser language to determine which language to use when showing search results or advertisements." (See www.google.com/privacy.html).

If you add the Google Toolbar to your Windows browser, then it can send Google information about the pages you view, and Google can update the Toolbar code automatically, without asking you. However, you can disable the Toolbar's "advanced features" by going to the Google menu and selecting privacy information. And it isn't "spyware" because Google isn't collecting information to sell, just to provide you with better searches.

People could also get better results simply by improving their search techniques. Few bother, which is a pity, because fruitless searches waste a lot of time. If you make more than a dozen searches a day, then a small improvement in your techniques can deliver dramatic benefits. With that in mind, here are my top 10 search tips.


Imagine what you want
It may sound obvious, but you have to search Google for the words that will be on the page you want, not for a description of the page or website. For example, if you wanted to find a comparative review of various PDAs, then - using the convention that anything inside square brackets is what you would type into Google - you could search for [comparative review of pdas]. The alternative is to imagine the sort of review you want. It will probably include the words Palm, Pocket PC, iPaq and Clie, so instead, try searching for [review palm pocket pc ipaq clie].


Use quotation marks
If you search for, say, [John Adams], Google will find all the pages with John and all the pages with Adams, even if the words are unconnected. This finds 3.6m hits. However, if you put the words in quotation marks, this tells Google to treat them as one unit. Using ["John Adams"] eliminates 3m hits. It is especially important to use quotes if you are looking for something that includes a "stop word". These are the words Google ignores, because they are too common. They include: a, about, are, at, by, from, I, in, of, how, la, that, the, this, to, will, who, what, where, and when. If you search for the band [the smiths] then Google will ignore "the," the stop word, so it is better to search for ["the smiths"]. However, if your search only contains stop words, Google will search for them, though ["the who"] still works better than [the who].


Use the + sign
Another way to make sure Google includes a particular word in its search is to put a plus sign in front of it.

Use the - sign
The plus sign adds a word to a search so using a minus sign takes one away. This is very useful as a way of eliminating lots of hits you don't want. I frequently search for technical information on stupidly difficult things such as transferring files from a MiniDisc player to a PC, and often get deluged with results from shopping and price comparison sites such as Dealtime, Kelkoo and Bizrate. Many of these can be eliminated by adding -merchant to the search term.


Try a wild card
Some experienced searchers don't like Google because they think it doesn't allow them to exploit hard-won skills in creating Boolean searches using "wild cards" and AND and OR commands (see below). But Google understands more than it often lets on. For example, suppose you want to find a number of quiz sites that decide what kind of flower, bird, geek or tin-pot dictator you are. You will probably be surprised to hear that searching for ["what * am I"] will do that, with the asterisk acting as a "wild card" for any word. You can also use two or more asterisks together for longer phrases. Searching for ["from * to * pc"] can be useful, and wild cards are not counted in the 10-word search allowance.


Use the site: command
Look at a page of Google results and you should notice that some hits are indented. This is because many sites would produce thousands of hits for a term, but Google shows only two from each site. It indents the second result and adds a link that offers "More results from" that site. For example, search for ["nathan milstein"] then scroll down and click on the link for "more results link for classical.onino.co.uk". This restricts the hits to that site. Now if you look in the search box, you will see that it says site:classical.onino.co.uk. This is the site: command, and you can type it in directly to search any site you like. It helps, of course, if the site has a short name, such as imdb.com [tampopo dvd site:imdb.com]. The neat thing is that you don't have to use a whole site name: you can search or exclude whole domains. For example, you can search for [tampopo dvd site:co.uk] or [tampopo dvd -site:com].


Use the operators
The site: operator is one of a long list that Google understands. These include filetype: (eg doc or pdf), intext: and allintext:, intitle: and allintitle:, inurl: and allinurl:, author: (in Google Groups) and location: (in Google News). What is the rest of the world saying about Beagle 2? Search for [beagle+2 -location:uk] to find out.

Google also understands a logical OR, as long as it is in caps. This means you can search for a hotel in Leeds OR Bradford, for example. It is very useful when people, places or things have alternative or variable spellings: [outsourcing bombay OR mumbai]. The OR command can be shortened to a vertical bar (|), as in [outsourcing bombay | mumbai]. Another way of adding alternatives is to use a twidde or tilde character (~). Thus if you search for [~food], Google also searches for cooking, cuisine, nutrition, recipes and restaurants.


The Advanced Search page
Fortunately, you don't have to learn all these special operators to use them. All you have to do is click on Google's Advanced Search link. This brings up a form with drop-down menu choices that lets anyone make complex searches without even thinking about it. This page includes options to search a particular period or pages in a specified language.


Other enhanced searches
Google is always adding new features, and as well as being a search engine, it also works as a dictionary (define, a glossary , and a very powerful calculator. It can even work out [the answer to life, the universe and everything]. But Google has also opened up its programming interface (API) so that other people can create applications to search its database of web pages. So far, most of these experiments are not very useful, but you can search recently added pages at GooFresh and compare results for keywords at GoogleFight. For more examples, see www.voelspriet.nl/googletools.htm.


Try a different search engine
Google is wonderful, there's no doubt about that. However, it does not always find the pages you want, so it is just as well to keep some alternatives handy. The main ones include stalwarts Alta Vista and All The Web, plus Vivisimo Vivisimo.com and Teoma. There are also "metasearch" search engines such as Dogpile and Metacrawler, which will send your query to several search engines at once. Google knows you have a choice, and it doesn't hurt to exercise it from time to time.

[DVTimes]

Review of google

http://www.searchengineshowdown.com/...le/index.shtml


Google has become for many the pre-eminent Web search engine. In Feb. 1999 it moved from Alpha test version to Beta and on Sept. 21, 1999 it officially launched.


Since that time it has made its mark with its relevance ranking based on link analysis, cached pages, and aggressive growth. Since its beta release, it has had phrase searching and the - for NOT, but it did not add an OR operation until Oct. 2000. In Dec. 2000, it added title searching. In June 2000 it announced a database of over 560 million pages, which grew to over 600 million by the end of 2000 and then 1.5 billion in Dec. 2001. The 2+ billion reported on their home page as of April 2002 includes indexed pages, unindexed URLs, and other file formats. By Nov. 2002, they moved their claim up to 3 billion. Use the table of contents on the left to navigate this review.

Databases: Google offers its own database of indexed Web pages along with another collection of URLs that it has not indexed such as duplicate URLs, redirected URLs, pages protected by a robots.txt file, and pages with access restrictions. Results from this latter database are relatively rarely seen but can be identified by the lack of an extract, size, and cached copy information in the results. Its regular index was expanded in 2001 to include additional file types: first PDF files, and then .ps, .doc, .xls, .txt, .ppt, .rtf, .asp, .wpd, and more. See Google Database Components for more details. Google has an image database and a Usenet news database known as Google Groups. Their News Search, Catalog Search, and Froogle (a shopping search) were all still in beta as of Aug. 2003.

Google also has a PageRank version of the Open Directory, and above their regular results, hits from their own news headlines database, stock quotes, and a phone number database may display. In addition it offers several specialized subsets: a government database of the .gov and .mil sites; University searches; a Linux search; an Apple/Macintosh search; and a Microsoft search.

The Google database is used by AOL, iWon, at Netscape's Search site, as the back-end search engine at Yahoo!, and the Weather Underground among many others. Yahoo! switched from Inktomi to Google in July 2000 and reaffirmed and more closely integrated Google results in Oct. 2002. BBCi used Google from May 2002 until March 2003 when they switched to Inktomi.

Strengths:
* Size and scope: It is now the largest, and includes PDF, DOC, PS, and many other file types
* Relevance based on sites' linkages and authority
* Cached archive of Web pages as the looked were indexed
* Additional databases: Google Groups, News, Directory, etc.

Weaknesses: See also the Google Inconsistencies Page
* Limited search features: no nesting, no truncation, does not support full Boolean
* Link searches must be exact and are incomplete
* Only indexes first 101 KB of a Web page and about 120 KB of PDFs
* Site clustering is difficult to turn off

Default Operation:
Multiple search terms are processed as an AND operation by default. Phrase matches are ranked higher

Boolean Searching:
Google uses an automatic Boolean AND between terms and has slowly been moving towards more Boolean support; however, it does not yet support the AND operator, NOT operator, or full Boolean searching with the ability to nest operators. In Feb. 1999, Google added the - symbol to perform a NOT function. In Oct. 2000, they added the ability to use an OR (which must be in upper case) to do some Boolean OR operations. See the Boolean Searching on Google page for more details on how to get Google to do certain kinds of Boolean searches.

The + used to be able to be used to require a term, but since the default operation was AND, the + was never really needed and for a while caused the following message to appear:


Google always searches for pages containing all the words in your query, so you do not need to use + in front of words.
However, the + can be used for forcing a search on stop words.

Proximity Searching:
In Feb. 1999, Google added phrase searching designated in the usual manner by enclosing the phrase in "double quotes." Google also detects phrase matches even when the quotes are not used and usually ranks phrase matches higher. No other proximity searching is directly available. However, using the wildcard word within a phrase trick described below, the unofficial Google API Proximity Search tool can reproduce proximity searching up to a distance of 3 words.

Truncation:
No truncation is available nor is there any automatic plural searching, word stemming, or other related features. However, within phrases, there is a trick which can be used for a wildcard word. Use an asterisk * within a phrase search to match any word in that position. So, for example, to find "a little neglect may breed mischief" when you are not sure of the second to last word, search "a little neglect may * mischief". Multiple asterisks can be used as in "a little * * * mischief". This is the only way Google supports a wildcard symbol.

While not exactly truncation, the synonym operator of a tilde ~ before a search term, with no space, to tell Google to look for synonyms. So a search on yosemite ~trails will find pages that have terms like 'hiking,' 'rides,' and 'maps.' This synonym finder will sometimes include plural, singular, or other grammatical variants as well. So the earlier search also found matches with 'trail' and 'trailer.' So the ~ can be used to get something a bit closer to truncation but not very. Bear in mind that the ~ only works in Google's Web database and only for English language terms.

Case Sensitivity:
Google has no case sensitive searching. Using either lower or upper case results in the same hits.

Field Searching:
Google offers several field searches connected with entering URLs. In the December 2000 revision of its advanced search form, it add several title and URL field searches.

Note that most field searching cannot be combined with other query words. In others words, a search entered such as uniqueword link:name.com will only be processed as if only the field search was present as in link:name.com. The uniqueword is ignored. One exception is the site: limit which cannot be used alone and must have another search term. The more recently added intitle: and inurl: fields can be combined with other search terms.

Field Explanation
intitle: Finds pages that have the term(s) in the HTML title element. Can be combined with other search terms. intitle:search engines. This should find 'search' in the title and 'engines' anywhere in the page.
inurl: Finds pages that have the term(s) somewhere in the URL (host name, path, or filename). Can be combined with other search terms. inurl:searchenginewatch.
allintitle: Finds pages that have the term(s) in the HTML title element. allintitle:search engines.
link: Finds pages which contain hypertext links to the exact specified URL. link:notess.com/search finds pages with links to this site.
allinurl: Finds pages that have the term(s) somewhere in the URL (host name, path, or filename). allinurl:searchenginewatch.
site: Finds pages from the designated Web site. Paths and file names cannot be included. An additional search term must be used. Try a term from the domain name for the most comprehensive results. notess site:notess.com finds how many pages Google has index or listed. As of April 2000, this was an undocumented search feature.
allinanchor: Finds pages that have the term(s) somewhere in the links to the page. .
related: Invokes GoogleScout to find other pages similar in linkage patterns to the given URL and at a similar hierarchical level. The URL must be exact. In other works related:notess.com and related:www.notess.com find different results.
flink: Used to find pages linked from the given URL. No longer working as of Oct. 30, 1999. flink:notess.com

Before the official release in Sept. 1999, clicking the small bar graph at the beginning of a displayed hit would automatically run a link: search, but that graphic disappeared with the official launch. Another field search which can be used is related:[URL] which invokes GoogleScout to find other pages similar in linkage patterns to the given URL.

Limits:
Google has language, domain, date, filetype, and adult content limits. The date limit, added in July 2001, is only available on the Advanced Search page. Only three options are available: Past 3 Months, Past 6 Months, or Past Year.

The file type limit, added along with the addition of other file types to the Google index, was added to the Advanced Search page in Nov. 2001. The Advanced Search page only offers file type limits under the label of File Formats for PDF, Word (.doc), Excel (.xls), PowerPoint (.ppt), and Rich Text Format (.rtf). Using the filetype: prefix, the file type limit can also be used for PostScript (.ps), Text (.txt), .htm, WordPerfect (.wpd), and other file extensions. To use the prefix command, just put the extension immediately after filetype: as in differentials filetype:ps.

[DVTimes]

Google introduced the language limit in April 2000 with eleven languages which was expanded as of Aug. 2000 to 24. As of July 2001, Russian was added. In Nov. 2001, Arabic and Turkish and then in early 2002 Catalan, Croatian, Indonesian, Serbian, Slovak, and Slovenian joined the group for the following 34 language limit options. These are available on the Advanced Search page and their Language Tools page.

Arabic
Bulgarian
Catalan
Chinese (Simplified & Traditional)
Croatian
Czech
Danish
Dutch
English
Estonian
Finnish
French
German
Greek
Hebrew
Hungarian
Icelandic
Indonesian
Italian
Japanese
Korean
Latvian
Lithuanian
Norwegian
Polish
Portuguese
Romanian
Russian
Serbian
Slovak
Slovenian
Spanish
Swedish
Turkish
To choose more than one at a time use the preferences page, which also offers a choice for which of 14 languages the surrounding text will be displayed in.

In May 2000, a family filter was added which tries to exclude adult Web pages. Turn it on from the preferences page.

The Advanced Search offers a domain limit, which can be used to limit results to those from the specified domain or it can be used to exclude results from a specified domain.

Stop Words:
Google does ignore frequent words. Its documentation mentions terms such as 'the', 'of', 'and', and 'or'. However, it also notes that these can be searched by putting + in front of them. As of March 2000, 'the' was a stop word that could not be searched even with the + sign. But by 2002, 'the' could be searched with the plus. Be sure to only place the + in front of stop words. If a + is placed in front of a non-stop word in the same query, all + signs will be ignored. As of Nov. 2001, stop words within a phrase no longer require a + sign and will automatically be searched. Also, if only stop words are entered even without phrase markings, they will be searched.

Sorting:
Results are sorted by relevance which is determined by Google's PageRank analysis, determined by links from other pages with a greater weight given to authoritative sites. Pages are also clustered by site. Only two pages per site will be displayed, with the second indented. Others are available via the [ More results from . . . ] link. If the search finds less than 1,000 results when clustered with two pages per site and if you page forward to the last page, after the last record the following message will appear:


In order to show you the most relevant results, we have omitted some entries very similar to the 63 already displayed. If you like, you can repeat the search with the omitted results included.
Clicking the "repeat the search" option will bring up more pages, some of which are near or exact duplicates of pages already found while others are pages that were clusted under a site listing. However, clicking on that link will not necessarily retrieve all results that have been clustered under a site. You can also just add &filter=0 to the end of a search results URL. To see all results available on Google, you need to check under each site cluster as well as using the "repeat this search" option.

Display:
The display includes the title, URL, a brief extract showing text near the search terms, the file size, and for many hits, a link to a cached copy of the page. This cached copy is from Google's index and may be older than the version currently available on the Web. The cached copy will display highlighted search terms. If more than one search term is used, each has a different color highlighting. The default output is 10 hits per screen, but the searcher can also choose 20, 30, 50, or 100 hits at a time on the preferences page. In June 1999, numeric relevance scores and "phase match" or "partial phrase match" indicators were removed. In Sept. 1999, the graphic relevancy bar with its link to a link: search was removed. At the same time, a GoogleScout link was added. GoogleScout is now just labeled as "Similar pages" and find other pages similar in linkage patterns to the displayed hit. In April 2000, Google started clustering results by site. Formerly, hits from the same site would be listed indented under the first. As of April 2000, only the first two hits are displayed (with the second one indented) and the rest available under a
[ More results from hostname ]
link.

With the addition of non-HTML files in 2001, Google added two notes to the display to identify those files. Before the title in the first line of the display, [PDF] or [PS] or [XLS] is used to denote the different file format. On some, a second line of the display lists
File Format: PDF/Adobe Acrobat - View as Text.

Around Aug. 2001, Google started refreshing the indexing of certain pages (those with daily updates) more frequently than the rest of the database. These were marked with "Fresh!" after the URL and size. In Dec. 2001, this tag was changed to list the indexing date. As of Feb. 2002, 3 million pages were being refreshed on an almost daily basis.

Unique: Google was the first general search engine that provides access to pages at the time they were indexed, designated as "cached" pages. For an alternative sources for cached pages see the archives page. Google is also the only search engine that searches for some characters. As of Sept. 2003, it would search for the ampersand & and the underscore _ characters by themselves or as part of a character hahahahahahahaha In other words, a search on adv_search gets different results than "adv search" and &tc differs from tc. While it would not search # or + in most cases, it does differentiate c#, c++, c+, and c. It does not, however, differentiate c*, c+@, or c+-, interpreting c* as c and both c+- and c+@ as c+. (These c+ type strings are all various programming languages.) Other punctuation marks may change the sorting of results.